Lens virtual entrance pupil fitting and sampling method, device, equipment and storage medium

Abstract

This application specifically relates to a method, apparatus, device, and storage medium for fitting and sampling a virtual entrance pupil of a lens. The method includes: generating multiple ray sampling points on an initial plane based on preset lens parameters and an initial field of view; performing ray tracing on these multiple ray sampling points to obtain effective ray sampling points; scanning the effective ray sampling points to identify semi-critical rays and determine their exploration direction; calculating the initial point orientation offset of the semi-critical ray using Newton's iterative formula based on the semi-critical ray and its exploration direction; determining the virtual entrance pupil boundary point corresponding to the semi-critical ray when the initial point orientation offset is less than a preset convergence error; fitting the virtual entrance pupil boundary point of the semi-critical ray using radial basis functions to obtain the effective virtual entrance pupil region; and selecting and determining a triangular mesh set for ray tracing. This solves the problems of inaccurate virtual entrance pupil boundary calculation and low sampling efficiency.

Description

Technical Field

[0001] This application relates to the field of optical system technology, and in particular to a method, apparatus, device and storage medium for fitting and sampling a virtual entrance pupil of a lens. Background Technology

[0002] In cutting-edge fields such as end-to-end optics-algorithm joint optimization, Monte Carlo rendering based on physical camera models, and modern complex lens design, a key preliminary step is to accurately determine and efficiently sample the virtual entrance pupil of the optical system. The boundary of the virtual entrance pupil is determined by the trajectory of the "critical rays" on the virtual entrance pupil plane, which are precisely blocked or restricted during propagation by the physical boundary of an optical element in the system (such as the edge of a mirror or an aperture stop).

[0003] In related technologies, method (1) is a geometric simplification method, which approximates the virtual entrance pupil as a circle or ellipse, and then performs uniform or low-difference sequence sampling within it. Method (2) is a dense sampling ray tracing and culling method, which generates a large number of rays in a sufficiently large area on the initial plane, performs complete physical path tracing for each ray, and then judges and culls all invalid rays by whether the rays reach the image plane, retaining the set of valid rays. Method (3) is a boundary search method, which uses radial or polar coordinate grid scanning, combined with numerical search algorithms such as the bisection method, to gradually approach and locate the boundary points of the virtual entrance pupil in multiple directions.

[0004] However, the methods in related technologies make overly strong geometric assumptions, leading to boundary distortion; they suffer from huge computational redundancy, resulting in low sampling efficiency; and their boundary judgments are discrete, making the methods non-differentiable. Furthermore, they lack a unified and efficient data structure, which urgently needs to be addressed. Summary of the Invention This application provides a method, apparatus, device, and storage medium for fitting and sampling a virtual entrance pupil of a lens, in order to solve the problems of inaccurate virtual entrance pupil boundary calculation and low sampling efficiency.

[0005] The first aspect of this application provides a method for fitting and sampling a virtual entrance pupil of a lens, comprising the following steps: Based on preset lens parameters and initial field of view, multiple ray sampling points are generated on the initial plane, and ray tracing is performed on the multiple ray sampling points to obtain effective ray sampling points; The effective light sampling points are scanned to identify the semi-critical light rays in the effective light sampling points and to determine the exploration direction of the semi-critical light rays. Based on the semi-critical ray and its exploration direction, the initial point orientation offset of the semi-critical ray is calculated using Newton's iterative formula. When the initial point orientation offset is less than the preset convergence error, the virtual entrance pupil boundary point corresponding to the semi-critical ray is determined. The effective virtual entrance pupil region is obtained by fitting the virtual entrance pupil boundary points of the semi-critical ray with radial basis functions, and the effective virtual entrance pupil region is screened to determine a triangular mesh set, and ray tracing is performed based on the triangular mesh set.

[0006] Optionally, the step of performing ray tracing on the plurality of ray sampling points to obtain effective ray sampling points includes: Determine the coordinates of each light sampling point; Based on the coordinates of each ray sampling point and the initial field of view, the initial direction of each ray sampling point is determined, and the ray equation is constructed according to the initial direction of each ray sampling point; Based on the preset surface equation and the light equation, the intersection points of the light rays generated by each light sampling point according to the corresponding initial direction with multiple lens surfaces are calculated to obtain multiple intersection points; Based on the preset Snell's law, the refraction direction of the light ray corresponding to each light sampling point at the corresponding intersection point is calculated, and the effective light ray reaching the preset image plane after refraction is determined. The intersection point of the effective light ray with the multiple lens surfaces is taken as the effective light sampling point.

[0007] Optionally, the Newton iteration formula is: ; in, This is the optimal value after iterating the directional offset. For directional offset, For value function, It is the first derivative of the value function with respect to the orientation offset.

[0008] Optionally, the radial basis function is: ; in, For the fitting function, For discrete virtual entrance pupil boundary points, For the first The center points of the basis functions For the corresponding weights, For radial basis functions, The distance between the virtual entrance pupil boundary point and the center point of the basis function is the Euclidean distance.

[0009] Optionally, the step of filtering the effective virtual entrance pupil region to obtain a triangular mesh set includes: Calculate the minimum bounding rectangle of the effective virtual entrance pupil region; The minimum bounding rectangle is regularly subdivided according to the preset basic unit to obtain multiple initial triangular meshes; Based on the positional relationship of the multiple initial triangular meshes, target triangular meshes located inside the effective virtual entrance pupil region and / or intersecting with the fitted curve of the effective virtual entrance pupil region are selected, and the triangular unit set is constructed based on the target triangular meshes.

[0010] A second aspect of this application provides a fitting and sampling device for a lens virtual entrance pupil, comprising: The tracking module is used to generate multiple ray sampling points on an initial plane based on preset lens parameters and an initial field of view, and to perform ray tracing on the multiple ray sampling points to obtain effective ray sampling points; The scanning module is used to scan the effective light sampling points, identify the semi-critical light rays in the effective light sampling points, and determine the exploration direction of the semi-critical light rays; The calculation module is used to calculate the initial point orientation offset of the semi-critical ray based on the semi-critical ray and the exploration direction of the semi-critical ray using Newton's iterative formula. When the initial point orientation offset is less than the preset convergence error, the virtual entrance pupil boundary point corresponding to the semi-critical ray is determined. The determination module is used to fit the virtual entrance pupil boundary points of the semi-critical ray with radial basis functions to obtain an effective virtual entrance pupil region, filter the effective virtual entrance pupil region to determine a triangular mesh set, and perform ray tracing based on the triangular mesh set.

[0011] Optionally, the tracking module is specifically used for: Determine the coordinates of each light sampling point; Based on the coordinates of each ray sampling point and the initial field of view, the initial direction of each ray sampling point is determined, and the ray equation is constructed according to the initial direction of each ray sampling point; Based on the preset surface equation and the light equation, the intersection points of the light rays generated by each light sampling point according to the corresponding initial direction with multiple lens surfaces are calculated to obtain multiple intersection points; Based on the preset Snell's law, the refraction direction of the light ray corresponding to each light sampling point at the corresponding intersection point is calculated, and the effective light ray reaching the preset image plane after refraction is determined. The intersection point of the effective light ray with the multiple lens surfaces is taken as the effective light sampling point.

[0012] Optionally, the Newton iteration formula is: ; in, This is the optimal value after iterating the directional offset. For directional offset, For value function, It is the first derivative of the value function with respect to the orientation offset.

[0013] Optionally, the radial basis function is: ; in, For the fitting function, For discrete virtual entrance pupil boundary points, For the first The center points of the basis functions For the corresponding weights, For radial basis functions, The distance between the virtual entrance pupil boundary point and the center point of the basis function is the Euclidean distance.

[0014] Optionally, the determining module is specifically used for: Calculate the minimum bounding rectangle of the effective virtual entrance pupil region; The minimum bounding rectangle is regularly subdivided according to the preset basic unit to obtain multiple initial triangular meshes; Based on the positional relationship of the multiple initial triangular meshes, target triangular meshes located inside the effective virtual entrance pupil region and / or intersecting with the fitted curve of the effective virtual entrance pupil region are selected, and the triangular unit set is constructed based on the target triangular meshes.

[0015] A third aspect of this application provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform a lens virtual entrance pupil fitting and sampling method as described in the above embodiments.

[0016] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the lens virtual entrance pupil fitting and sampling method as described in the above embodiments.

[0017] A fifth aspect of this application provides a computer program product storing a computer program that, when executed by a processor, implements the lens virtual entrance pupil fitting and sampling method as described in the above embodiments.

[0018] Therefore, this embodiment generates multiple ray sampling points on an initial plane based on preset lens parameters and an initial field of view. Ray tracing is then performed on these multiple ray sampling points to obtain valid ray sampling points. These valid ray sampling points are scanned to identify semi-critical rays and determine their exploration direction. Based on the semi-critical rays and their exploration direction, Newton's iterative formula is used to calculate the initial point orientation offset of the semi-critical ray. When the initial point orientation offset is less than a preset convergence error, the virtual entrance pupil boundary point corresponding to the semi-critical ray is determined. A radial basis function is used to fit the virtual entrance pupil boundary point of the semi-critical ray to obtain the effective virtual entrance pupil region, and a triangular mesh set is selected for ray tracing. This solves the problems of inaccurate virtual entrance pupil boundary calculation and low sampling efficiency.

[0019] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0020] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a method for fitting and sampling a virtual entrance pupil of a lens according to an embodiment of this application; Figure 2 Schematic diagrams of spherical, aspherical, and freeform lenses for a lens virtual entrance pupil fitting and sampling method provided according to an embodiment of this application; Figure 3 This is a schematic diagram illustrating the screening of semi-critical rays and the generation of corresponding exploration directions in a lens virtual entrance pupil fitting and sampling method according to an embodiment of this application; Figure 4 This is a schematic diagram illustrating the effect of virtual entrance pupil boundary point fitting and region triangulation in a lens virtual entrance pupil fitting and sampling method provided according to an embodiment of this application. Figure 5 This is a schematic diagram comparing the virtual entrance pupil region triangulation filling result of a lens virtual entrance pupil fitting and sampling method according to an embodiment of this application with the coverage of traditional 256×256 dense sampling points; Figure 6 This is a schematic diagram showing the completion of a lens virtual entrance pupil fitting and sampling method according to an embodiment of this application, comparing the virtual entrance pupil region fitting with the bisection method and the traditional dense sampling method in terms of average execution time. Figure 7This is a schematic diagram comparing the coverage and redundancy of the virtual entrance pupil region RBF triangulation fitting with traditional elliptical fitting and rectangular fitting under different fields of view, according to an embodiment of the present application, using a lens virtual entrance pupil fitting and sampling method. Figure 8 This is a flowchart illustrating the accurate solution and efficient sampling method for fitting and sampling a lens virtual entrance pupil according to an embodiment of this application; Figure 9 This is a schematic diagram of a lens virtual entrance pupil fitting and sampling device provided according to an embodiment of this application; Figure 10 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0021] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0022] Before introducing the lens virtual entrance pupil fitting and sampling method of the embodiments of this application, let's briefly introduce the lens virtual entrance pupil fitting and sampling method in related technologies.

[0023] Specifically, in this invention, the virtual entrance pupil is defined as a region, typically located on a virtual plane in front of the first mirror of the optical system. The virtual entrance pupil is defined by the trajectory of critical rays that are blocked or restricted during propagation by the physical boundary of an optical element in the system (such as the edge of a mirror or an aperture stop). All light rays emanating from this region and having a specific direction can successfully pass through the entire optical system, which consists of multiple sets of mirrors and aperture stops, and ultimately converge on the image plane to form an effective image.

[0024] Furthermore, in traditional optical systems or under small field-of-view conditions, the aperture stop is the primary factor restricting the passage of light, and the shape of the virtual entrance pupil is usually close to a circle. However, as optical designs have become more complex, incorporating larger fields of view, freeform surfaces, and aspherical surfaces, the physical edges of the lens have begun to significantly obstruct off-axis light rays; this phenomenon is called vignetting. These factors work together to cause the boundary of the virtual entrance pupil to exhibit a complex and irregular geometric shape.

[0025] To obtain a virtual entrance pupil and perform light sampling, the following methods are commonly used: (1) Geometric simplification method: approximate the virtual entrance pupil as a circle or ellipse, and then perform uniform or low-difference sequence sampling inside it.

[0026] (2) Dense sampling ray tracing and elimination method: Generate a large number of rays in a sufficiently large area on the initial plane, perform complete physical path tracing for each ray, and then judge and eliminate all invalid rays by whether the ray reaches the image plane, and retain the set of valid rays.

[0027] (3) Boundary search method: Radial or polar coordinate grid scanning is used, combined with numerical search algorithms such as the bisection method, to gradually approach and locate the boundary point of the virtual entrance pupil in multiple directions.

[0028] However, while the above methods can provide preliminary results in simple scenarios, their inherent limitations become very prominent when faced with modern, complex optical systems, mainly in the following aspects: (1) Overly strong geometric assumptions lead to boundary distortion: The circular or elliptical boundary fitted by the geometric simplification method differs greatly from the real, irregular virtual entrance pupil boundary. This results in inaccurate sampling areas.

[0029] (2) The computational redundancy is huge, resulting in low sampling efficiency: Dense sampling ray tracing and elimination methods need to sample and track the entire link of light rays in a region that is far larger than the actual virtual entrance pupil area. Most of the light rays are blocked during the tracking process, and the computational cost is completely wasted, resulting in extremely low sampling efficiency. In order to achieve sub-pixel accuracy, the boundary search method also requires extremely dense grid scanning and multiple iterations, resulting in high computational complexity.

[0030] (3) Discrete boundary judgment leads to non-differentiable method: Whether it is Boolean elimination in dense sampling or boundary judgment in search method, they are essentially discrete and discontinuous operations. This makes the entire process of determining the virtual entrance pupil non-differentiable. Therefore, it cannot be embedded into an end-to-end differentiable optimization pipeline that requires gradient information for backpropagation, hindering the automatic and efficient joint optimization of lens parameters, sensor parameters, etc. based on gradient.

[0031] (4) Lack of unified and efficient data structures: The results obtained by related technologies are usually discrete sets of boundary points or regions defined by Boolean masks. When these data structures are used for subsequent sampling and rendering, additional format conversion or complex region judgment is often required, which increases the coupling and complexity of engineering implementation, and makes it difficult to guarantee the uniform distribution of sampling points, especially near irregular boundaries.

[0032] This application addresses the aforementioned problems by proposing a fitting and sampling method for a lens's virtual entrance pupil. In this method, based on preset lens parameters and an initial field of view, multiple ray sampling points are generated on an initial plane. Ray tracing is then performed on these multiple ray sampling points to obtain effective ray sampling points. These effective ray sampling points are scanned to identify semi-critical rays and determine their exploration directions. Based on the semi-critical rays and their exploration directions, the initial point orientation offset of the semi-critical ray is calculated using Newton's iterative formula. When the initial point orientation offset is less than a preset convergence error, the virtual entrance pupil boundary point corresponding to the semi-critical ray is determined. A radial basis function is used to fit the virtual entrance pupil boundary point of the semi-critical ray to obtain the effective virtual entrance pupil region, and a triangular mesh set is selected for ray tracing. This solves the problems of inaccurate virtual entrance pupil boundary calculation and sampling efficiency. Specifically, Figure 1 This is a schematic flowchart illustrating a method for fitting and sampling a virtual entrance pupil of a lens, as provided in an embodiment of this application.

[0033] like Figure 1 As shown, the fitting and sampling method for the virtual entrance pupil of the lens includes the following steps: In step S101, based on preset lens parameters and initial field of view, multiple ray sampling points are generated on the initial plane, and ray tracing is performed on the multiple ray sampling points to obtain effective ray sampling points. Specifically, based on the lens parameters (preset lens parameters) and initial ray sampling, forward physical ray tracing is performed to calculate and record the intermediate information and validity of each ray. On the initial plane ( Generate ray sampling points using a regular grid. ,in, The coordinates of the light sampling point , It is the initial direction of the light ray based on the field of view angle (initial field of view angle). Generally, a light ray can be represented as: ;in, This is the forward travel distance from the current ray intersection point to the next ray intersection point. Subsequently, ray tracing is performed on each ray to determine the valid rays, and their corresponding initial sampling points are recorded as valid ray sampling points.

[0034] Optionally, in some embodiments, ray tracing is performed on multiple ray sampling points to obtain effective ray sampling points, including: determining the coordinates of each ray sampling point; determining the initial direction of each ray sampling point based on the coordinates and initial field of view of each ray sampling point, and constructing a ray equation according to the initial direction of each ray sampling point; calculating the intersection points of the ray generated by each ray sampling point according to the corresponding initial direction with multiple lens surfaces based on preset surface equations and ray equations to obtain multiple intersection points; calculating the refraction direction of the ray corresponding to each ray sampling point at the corresponding intersection point based on preset Snell's law, and determining the effective ray that reaches the preset image plane after refraction, and taking the intersection points of the effective ray with multiple lens surfaces as effective ray sampling points.

[0035] It is understandable that ray tracing can be divided into two steps: the first step is to calculate the intersection point of the ray with the surface. Surface equations are typically used to represent lens surfaces in optical systems. The two-dimensional lens layout diagram used in the embodiments of this application is shown below. Figure 2 As shown, Figure 2 This is a schematic diagram of spherical, aspherical, and freeform lenses for a lens virtual entrance pupil fitting and sampling method according to an embodiment of this application; wherein, the lens surface of the spherical lens can be described as: ; in, It is the vertical coordinate of a point on the surface. It's time to go. Distance between axes. It is the curvature of the spherical part.

[0036] The lens surface of an even-order aspherical lens can be described as follows: ; in, This is the conicity factor, and the subsequent term represents the deformation of the spherical portion. Is it an aspherical first Order coefficient.

[0037] Common freeform surfaces include XY polynomials, Zernike polynomials, B-splines, and NURBS, which are usually analytical expressions of higher-order terms. A Zernike polynomial freeform surface can be expressed as follows: ; Solve the surface equation and the ray equation simultaneously to find the ray travel distance from the current ray intersection point to the next ray intersection point. .

[0038] ; For spherical lenses, this equation exists. For analytical solutions, however, for aspherical or freeform lenses, numerical root-finding methods are needed to obtain high-precision approximate solutions.

[0039] The second step is to calculate the refraction of light rays at the surface of the medium. At the intersection... The normal vector at that point is the gradient of the lens surface: ; The normalized normal vector is: .

[0040] According to the pre-established Snell's law, the direction of the refracted light rays... It can be calculated using the following formula: ; in, , and These are the refractive indices of the material before and after refraction.

[0041] For the equation of light Differentiate: ; The intersection and direction Jacobian matrices are among them. and It can be accumulated during forward physical ray tracing. It can be solved by simultaneous equations Taking the partial derivative, we get: ; For the equation of refraction Differentiate: ; in, It is the normalized normal vector Jacobian matrix, which will be derived later. Differentiation yields : ; Normalized normal vector Jacobian matrix This can be derived from the chain rule: ; in It is the gradient rate of change of the lens surface, calculated by the following formula: ; in The Hessian matrix for the lens surface equation: .

[0042] Furthermore, the effective rays that reach the preset image plane after refraction are determined, and the intersections of the effective rays with multiple lens surfaces are taken as effective ray sampling points. Only the initial coordinates of the starting point on the initial plane of the rays that ultimately successfully reach the image plane are determined and recorded as effective ray sampling points.

[0043] In step S102, the effective light sampling points are scanned to identify the semi-critical light rays in the effective light sampling points and determine the exploration direction of the semi-critical light rays.

[0044] Among them, semi-critical rays refer to a special type of ray that is identified from all effective ray sampling points after preliminary ray tracing and screening, and is located near the boundary of the effective light region.

[0045] Specifically, such as Figure 3 As shown, Figure 3 This illustration shows a method for fitting and sampling a lens virtual entrance pupil according to one embodiment of this application, which involves screening semi-critical rays and generating corresponding exploration directions. This embodiment further analyzes and filters effective ray sampling points to determine semi-critical rays and generate corresponding exploration directions. First, a systematic scan of the ray sampling points is performed, in row and column order, ensuring that every ray is evaluated. During this process, the first and last valid ray sampling points in each row and column are recorded. These first and last valid ray sampling points identified during the scan are marked as the initial sampling points for semi-critical rays. These semi-critical rays are crucial for accurately fitting the virtual entrance pupil boundary, as they represent potential candidate points for the virtual entrance pupil boundary. For each ray marked as a semi-critical ray, an exploration direction is specified. The exploration direction can be horizontal (along...). (Axis) or vertical (along) (Axis), which depends on the position of the light.

[0046] In step S103, based on the semi-critical ray and its exploration direction, the initial point orientation offset of the semi-critical ray is calculated using Newton's iterative formula. When the initial point orientation offset is less than the preset convergence error, the virtual entrance pupil boundary point corresponding to the semi-critical ray is determined.

[0047] Optionally, in some embodiments, Newton's iteration formula is: ; in, This is the optimal value after iterating the directional offset. For directional offset, For value function, It is the first derivative of the value function with respect to the orientation offset.

[0048] The preset convergence error can be a threshold set by the user, a threshold obtained through a finite number of experiments, or a threshold obtained through a finite number of computer simulations; no specific limitation is made here. The virtual entrance pupil boundary points are a series of discrete spatial coordinate points located on the initial object plane in the imaging analysis of complex optical systems, constituting the precise contour of the actual light-transmitting region of the virtual entrance pupil.

[0049] Specifically, for each ray marked as a semi-critical ray, its initial sampling point is: Its exploration direction is Therefore, the boundary point of the virtual entrance pupil can be represented as: ; Solving for the boundary points of the virtual entrance pupil mainly involves solving for the orientation offset. From the above formula, we can see that .

[0050] In a small field of view, the aperture stop primarily restricts the amount of light that can pass through the optical system and ultimately reach the image plane. As the field of view increases, the edges of certain mirror surfaces may also affect the propagation of light, thus limiting the light's reach to the image plane. In this invention, for each semi-critical ray, the method involves calculating the squared distance difference from its intersection point with the lens surface or aperture stop to the corresponding edge. : ; in, For the first Net aperture of the optical surface For light and the first Intersection of optical surfaces Coordinates. Then the error. For the intersection point The gradient is: ; Error obtained from the chain rule Relative to offset The gradient is: ; in, yes of Quantity.

[0051] Assume that the entire system is linear within a small neighborhood of the initial sampling point of the semi-critical ray. This leads to an understanding of the error... The first-order Taylor expansion is: ; Setting the above equation to zero, we can obtain a first-order approximation of the offset. : ; To ensure accuracy based on the semi-critical ray and offset The resulting critical ray not only intersects precisely at the edge of a lens surface, but also continues to propagate to the image plane. This invention limits the critical ray offset corresponding to the surface. Select the minimum value This process allows for the precise determination of the confining surface. This refers to the lens surface or aperture stop that has a decisive influence on the path of light propagation. It should be noted that... The value is the squared difference of the distance from the intersection point of the light rays on the k-th optical surface to the edge of the lens surface.

[0052] Furthermore, this invention proposes a high-precision method for calculating virtual entrance pupil boundary points based on Newton's iteration method; this method repeatedly performs optical tracing and updates gradient information, and uses Newton's iteration formula to update the offset. This approximates the intersection of the critical ray and the lens surface or aperture stop, thereby accurately determining the boundary rays that constitute the virtual entrance pupil boundary. For the newly generated critical ray, if its offset... If the convergence error is less than the preset limit, or if the updated ray fails to reach the image plane because it just exceeds the edge of the optical element, the ray at the current iteration point is considered a boundary ray on the virtual entrance pupil boundary. Typically, most semi-critical rays can complete this process within 1 to 3 iterations.

[0053] In step S104, the effective virtual entrance pupil region is obtained by fitting the virtual entrance pupil boundary points of the semi-critical ray with radial basis functions, and the effective virtual entrance pupil region is screened to determine the triangular mesh set, and ray tracing is performed based on the triangular mesh set.

[0054] Optionally, in some embodiments, the radial basis function is: ; in, For the fitting function, For discrete virtual entrance pupil boundary points, For the first The center points of the basis functions For the corresponding weights, For radial basis functions, denoted as Euclidean distance between the virtual entrance pupil boundary point and the center point of the basis functions.

[0055] Specifically, such as Figure 4 As shown, Figure 4This diagram illustrates the effect of virtual entrance pupil boundary point fitting and region triangulation in a lens virtual entrance pupil fitting and sampling method according to an embodiment of this application. To accurately fit discrete virtual entrance pupil boundary points, the Radial Basis Function (RBF) method is employed. The RBF method is a powerful interpolation and approximation technique that approximates complex functions or surfaces by defining a set of basis functions. The RBF method is used to determine the effective virtual entrance pupil region; and the effective virtual entrance pupil region is then filtered to determine a triangular mesh set that precisely matches the virtual entrance pupil boundary. Ray tracing is then performed based on this triangular mesh set.

[0056] Optionally, in some embodiments, the effective virtual entrance pupil region is screened to obtain a triangular mesh set, including: calculating the minimum bounding rectangle of the effective virtual entrance pupil region; performing regular subdivision of the minimum bounding rectangle according to preset basic units to obtain multiple initial triangular meshes; and, based on the positional relationship of the multiple initial triangular meshes, selecting target triangular meshes located inside the effective virtual entrance pupil region and / or intersecting with the fitted curve of the effective virtual entrance pupil region, and constructing a triangular unit set based on the target triangular meshes.

[0057] Understandably, after obtaining the RBF fitting region, this embodiment calculates the minimum bounding rectangle of the region and regularly subdivides the rectangle using equilateral triangles (preset basic units) as basic units, thereby forming a uniform triangular mesh (initial triangular mesh). Subsequently, using a geometric determination algorithm, based on the positional relationship between the centroid and vertices of each triangle and the fitting curve, all triangles located inside the fitting region or intersecting the fitting curve are selected and retained (target triangular mesh), thus obtaining a set of triangular meshes that precisely fits the virtual entrance pupil boundary. The triangular mesh units obtained in this embodiment are all equilateral, with consistent areas, and their outlines completely coincide with the RBF fitting boundary. Rays can be projected once within each triangle using centroid sampling or the Monte Carlo method without additional boundary determination, thus avoiding the problems of uneven density, missed sampling, and redundancy that occur in the edge region in traditional uniform / random sampling, thereby significantly improving the sampling uniformity and computational efficiency in the ray tracing process.

[0058] To facilitate those skilled in the art to further understand the fitting and sampling method of the lens virtual entrance pupil in the embodiments of this application, the following is combined with... Figures 5 to 8 The embodiments shown will be described in detail.

[0059] Specifically, such as Figure 5 As shown, Figure 5This diagram illustrates a comparison between the virtual entrance pupil region triangulation filling result and the traditional 256×256 dense sampling point coverage of a lens virtual entrance pupil fitting and sampling method according to an embodiment of this application. For the virtual entrance pupil boundaries of three different lens types under three fields of view, this invention uses only about 200 equilateral triangles to fill the irregular virtual entrance pupil region. The coverage is then compared and verified with the traditional 256×256 dense sampling (65536 points) as the ground truth (GT). The results show that the triangulation region coverage of the ground truth points is ≥99.5% in all fields of view, reaching a maximum of 100.00%. The percentage of uncovered missed detection points is less than 0.5%, occurring only near the maximum field of view of the lens and concentrated in sub-pixel edge regions. This demonstrates that this invention, while reducing the number of sampling points by two orders of magnitude, still maintains accurate calculation of the virtual entrance pupil boundary, laying a solid foundation for subsequent efficient ray tracing.

[0060] Table 1

[0061] Furthermore, under the same hardware platform (Intel Core i7-12700K, 32 GB DDR4, Windows 11, Python 3.9.7) and 1000 repeated experiments, the average time consumption results of the method of this invention, the binary method, and the traditional dense sampling method (256×256) on multiple lens-field combinations are shown in Table 1 and Figure 6. Figure 6 This diagram illustrates a method for fitting and sampling a virtual entrance pupil of a lens according to an embodiment of this application. It compares the average execution time of virtual entrance pupil region fitting with the bisection method and the traditional dense sampling method. All tests were executed using eight CPU threads, with 10 pre-warm-up iterations performed to eliminate system cold start errors.

[0062] Furthermore, as shown in Table 1 and Figure 6As shown, for the spherical lens (lens 1), since the intersection of the ray and the spherical surface has an analytical closed solution, the present invention completes the boundary fitting in 0.046 s–0.061 s. Meanwhile, the average time of the bisection method is 0.155 s–0.170 s, and the average time of the dense sampling method is 0.252 s–0.257 s. For high-order aspherical lenses (lenses 2 and 3) requiring numerical root finding, the dense sampling method, due to the need to perform numerical root finding for each sampling point, has an average time consumption of 0.553s–0.685s. In comparison, the bisection method has an average time consumption of 0.228s–0.359s, which is an improvement over the dense sampling method, but still has a certain computational cost. The method of this invention, by selectively iteratively solving the "semi-critical rays" near the boundary, compresses the time consumption to 0.093s–0.127s, which not only ensures high accuracy of boundary fitting but also greatly improves computational efficiency, significantly outperforming the dense sampling method and the bisection method.

[0063] Furthermore, such as Figure 7 As shown, Figure 7 This diagram illustrates a comparison of the coverage and redundancy of the virtual entrance pupil region (RBF) triangulation fitting method with traditional elliptical and rectangular fitting methods under different viewing angles, representing an embodiment of this application. The results show that the RBF triangulation fitting method of the virtual entrance pupil region in this embodiment maintains nearly 100% coverage across all tested viewing angles, significantly outperforming the elliptical and rectangular fitting methods. Specifically, the coverage of elliptical and rectangular fitting decreases significantly at large viewing angles, while the method of this invention consistently covers all true value points. While maintaining high coverage, the method of this invention maintains a moderate level of redundancy, avoiding the introduction of invalid sampling areas, thereby significantly improving the overall efficiency of subsequent ray sampling and tracking.

[0064] Furthermore, this application embodiment can also provide an overview of the fitting and sampling method for the virtual entrance pupil of a lens through a specific embodiment.

[0065] Specifically, such as Figure 8 As shown, Figure 8 This is a flowchart illustrating the accurate solution and efficient sampling method for fitting and sampling a lens virtual entrance pupil according to an embodiment of this application. This application proposes an accurate solution and efficient sampling method for lens virtual entrance pupil, which realizes accurate fitting and efficient sampling of virtual entrance pupil in complex optical systems. It is particularly suitable for accurate fitting of virtual entrance pupil boundaries and efficient region filling sampling of lenses with multiple mirrors, large field of view, and freeform surfaces.

[0066] Furthermore, the efficient and accurate solution method for virtual entrance pupil boundary points in this invention transforms the geometric problem of boundary finding into a mathematical root-finding problem. First, ray tracing is used to initially screen and locate semi-critical rays near the boundary. Then, a differentiable ray tracing model is used to calculate the Jacobian matrix (gradient) of the distance (error function) between the ray trajectory and the edge of the limiting optical surface in the optical path relative to its initial position. The key advantage of this method lies in its wide applicability; as long as the optical surface acting as the limiting boundary (whether it's an aperture, lens, or mirror) has a differentiable mathematical expression, its gradient can be accurately obtained. Therefore, this technique can be extended to edge-limiting scenarios of arbitrarily complex curved lenses or aspherical mirrors. Finally, using Newton's iteration method with quadratic convergence speed, the gradient information is iteratively updated. Only a few steps are needed to quickly and accurately solve for the initial ray position that makes the error function zero, thus obtaining a virtual entrance pupil boundary point with sub-pixel accuracy. Its efficiency and accuracy far exceed those of traditional bisection methods or brute-force search. Furthermore, this method has good versatility; by tracing the light rays in reverse from the image side, the same technical framework can be directly applied to the accurate calculation of the virtual exit pupil boundary.

[0067] Furthermore, to efficiently sample virtual entrance pupil regions of arbitrarily complex shapes composed of discrete boundary points, this invention proposes an efficient triangular meshing and sampling method for irregular virtual entrance pupil regions. This method uses the RBF function to fit the discrete boundary points, generating a continuous mathematical boundary that accurately describes complex shapes, including non-convex and multi-connected regions. Subsequently, within the precise boundary defined by the RBF function, a final set of sampling points is generated by filling a uniformly sized equilateral triangular mesh. This method not only ensures that the sampling points perfectly fit the contour of the virtual entrance pupil, avoiding the large amount of redundant and invalid sampling generated by traditional bounding box methods, but also guarantees the uniform distribution of sampling points throughout the entire virtual entrance pupil region, thereby improving the efficiency and physical accuracy of subsequent ray tracing calculations.

[0068] According to the lens virtual entrance pupil fitting and sampling method proposed in this application embodiment, based on preset lens parameters and initial field of view, multiple ray sampling points are generated on the initial plane, and ray tracing is performed on each of the multiple ray sampling points to obtain effective ray sampling points. The effective ray sampling points are scanned to identify semi-critical rays and determine their exploration direction. Based on the semi-critical rays and their exploration direction, the initial point orientation offset of the semi-critical ray is calculated using Newton's iterative formula. When the initial point orientation offset is less than a preset convergence error, the virtual entrance pupil boundary point corresponding to the semi-critical ray is determined. The virtual entrance pupil boundary point of the semi-critical ray is fitted using a radial basis function to obtain the effective virtual entrance pupil region, and a triangular mesh set is selected for ray tracing. This solves the problems of inaccurate virtual entrance pupil boundary calculation and sampling efficiency. Next, referring to the accompanying drawings, a fitting and sampling device for a lens virtual entrance pupil according to an embodiment of this application is described.

[0069] Figure 9 This is a block diagram of the fitting and sampling device for the virtual entrance pupil of a lens according to an embodiment of this application.

[0070] like Figure 9 As shown, the fitting and sampling device 1000 for the virtual entrance pupil of the lens includes: a tracking module 100, a scanning module 200, a calculation module 300, and a determination module 400.

[0071] The tracking module 100 is used to generate multiple ray sampling points on the initial plane based on preset lens parameters and initial field of view, and to perform ray tracing on the multiple ray sampling points to obtain effective ray sampling points. The scanning module 200 is used to scan the effective light sampling points, identify the semi-critical light rays in the effective light sampling points, and determine the exploration direction of the semi-critical light rays; The calculation module 300 is used to calculate the initial point orientation offset of the semi-critical ray based on the semi-critical ray and the exploration direction of the semi-critical ray using Newton's iterative formula. When the initial point orientation offset is less than the preset convergence error, the virtual entrance pupil boundary point corresponding to the semi-critical ray is determined. The determination module 400 is used to obtain the effective virtual entrance pupil region by fitting the virtual entrance pupil boundary points of the semi-critical ray with radial basis functions, and to filter the effective virtual entrance pupil region to determine the triangular mesh set, and to perform ray tracing based on the triangular mesh set.

[0072] Optionally, the tracking module 100 is specifically used for: determining the coordinates of each ray sampling point; determining the initial direction of each ray sampling point based on the coordinates and initial field of view of each ray sampling point, and constructing a ray equation according to the initial direction of each ray sampling point; calculating the intersection points of the ray generated by each ray sampling point according to the corresponding initial direction with multiple lens surfaces based on the preset surface equation and ray equation to obtain multiple intersection points; calculating the refraction direction of the ray corresponding to each ray sampling point at the corresponding intersection point based on the preset Snell's law, and determining the effective ray of the refracted ray reaching the preset image plane, and taking the intersection point of the effective ray with multiple lens surfaces as the effective ray sampling point.

[0073] Alternatively, Newton's iterative formula is: ; in, This is the optimal value after iterating the directional offset. For directional offset, For value function, It is the first derivative of the value function with respect to the orientation offset.

[0074] Optionally, the radial basis functions are: ; in, For the fitting function, For discrete virtual entrance pupil boundary points, For the first The center points of the basis functions For the corresponding weights, For radial basis functions, denoted as Euclidean distance between the virtual entrance pupil boundary point and the center point of the basis functions.

[0075] Optionally, the determination module 400 is specifically used for: calculating the minimum bounding rectangle of the effective virtual entrance pupil region; performing regular subdivision of the minimum bounding rectangle according to preset basic units to obtain multiple initial triangular meshes; and selecting target triangular meshes located inside the effective virtual entrance pupil region and / or intersecting with the fitted curve of the effective virtual entrance pupil region based on the positional relationship of the multiple initial triangular meshes, and constructing a triangular unit set based on the target triangular meshes.

[0076] It should be noted that the explanation of the above-described embodiment of the fitting and sampling method for the virtual entrance pupil of the lens also applies to the fitting and sampling device for the virtual entrance pupil of the lens in this embodiment, and will not be repeated here.

[0077] According to the lens virtual entrance pupil fitting and sampling device proposed in this application embodiment, based on preset lens parameters and initial field of view, multiple ray sampling points are generated on an initial plane, and ray tracing is performed on each of the multiple ray sampling points to obtain effective ray sampling points; the effective ray sampling points are scanned to identify the semi-critical rays and determine the exploration direction of the semi-critical rays; based on the semi-critical rays and their exploration directions, the initial point orientation offset of the semi-critical rays is calculated using Newton's iterative formula; when the initial point orientation offset is less than a preset convergence error, the virtual entrance pupil boundary point corresponding to the semi-critical ray is determined; the virtual entrance pupil boundary point of the semi-critical ray is fitted using a radial basis function to obtain the effective virtual entrance pupil region, and a triangular mesh set is selected for ray tracing. Thus, the problems of inaccurate virtual entrance pupil boundary calculation and sampling efficiency are solved. Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 1001, the processor 1002, and the computer program stored on the memory 1001 and capable of running on the processor 1002.

[0078] When the processor 1002 executes the program, it implements the fitting and sampling method for the virtual entrance pupil of the lens provided in the above embodiments.

[0079] Furthermore, electronic devices also include: Communication interface 1003 is used for communication between memory 1001 and processor 1002.

[0080] The memory 1001 is used to store computer programs that can run on the processor 1002.

[0081] The memory 1001 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0082] If the memory 1001, processor 1002, and communication interface 1003 are implemented independently, then the communication interface 1003, memory 1001, and processor 1002 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized into address buses, data buses, control buses, etc. For ease of representation, Figure 10The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0083] Optionally, in a specific implementation, if the memory 1001, processor 1002, and communication interface 1003 are integrated on a single chip, then the memory 1001, processor 1002, and communication interface 1003 can communicate with each other through an internal interface.

[0084] The processor 1002 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0085] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for fitting and sampling the virtual entrance pupil of a lens.

[0086] This application also provides a computer program product, which stores a computer program that, when executed by a processor, implements the above-described method for fitting and sampling the virtual entrance pupil of a lens.

[0087] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0088] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0089] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0090] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0091] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

Claims

1. A method for fitting and sampling a virtual entrance pupil of a lens, characterized in that, Includes the following steps: Based on preset lens parameters and initial field of view, multiple ray sampling points are generated on the initial plane, and ray tracing is performed on the multiple ray sampling points to obtain effective ray sampling points; The effective light sampling points are systematically scanned in the order of rows and columns. The effective light sampling points at the beginning and end of each row and column are recorded. The effective light sampling points at the beginning and end of each row and column identified during the scanning process are marked as the initial sampling points of the semi-critical light and the exploration direction of the semi-critical light is determined. Based on the semi-critical ray and its exploration direction, the initial point orientation offset of the semi-critical ray is calculated. This calculation includes: calculating the distance error to the edge of each optical surface. , ,in, For the first Net aperture of the optical surface For light and the first Intersection of optical surfaces Coordinates, calculate the gradient of the error phase offset The initial point offset is calculated using a first-order Taylor expansion. Select the initial point offset Minimum surface As a target limiting surface; Repeatedly perform ray tracing to the surface The orientation offset is calculated using Newton's iterative formula, which is: ； in, This is the optimal value after iterating the directional offset. For value function, The virtual entrance pupil boundary point corresponding to the semi-critical ray is determined by the first derivative of the value function with respect to the orientation offset. The effective virtual entrance pupil region is obtained by fitting the virtual entrance pupil boundary points of the semi-critical ray with radial basis functions, and the effective virtual entrance pupil region is screened to determine a triangular mesh set, and ray tracing is performed based on the triangular mesh set.

2. The method according to claim 1, characterized in that, The step of performing ray tracing on the plurality of ray sampling points to obtain effective ray sampling points includes: Determine the coordinates of each light sampling point; Based on the coordinates of each ray sampling point and the initial field of view, the initial direction of each ray sampling point is determined, and the ray equation is constructed according to the initial direction of each ray sampling point; Based on the preset surface equation and the light equation, the intersection points of the light rays generated by each light sampling point according to the corresponding initial direction with multiple lens surfaces are calculated to obtain multiple intersection points; Based on the preset Snell's law, the refraction direction of the light ray corresponding to each light sampling point at the corresponding intersection point is calculated, and the effective light ray reaching the preset image plane after refraction is determined. The intersection point of the effective light ray with the multiple lens surfaces is taken as the effective light sampling point.

3. The method according to claim 1, characterized in that, The radial basis function is: ； in, For the fitting function, For discrete virtual entrance pupil boundary points, For the first The center points of the basis functions For the corresponding weights, For radial basis functions, The distance between the virtual entrance pupil boundary point and the center point of the basis function is the Euclidean distance.

4. The method according to claim 1, characterized in that, The process of filtering the effective virtual entrance pupil region to obtain a triangular mesh set includes: Calculate the minimum bounding rectangle of the effective virtual entrance pupil region; The minimum bounding rectangle is regularly subdivided according to the preset basic unit to obtain multiple initial triangular meshes; Based on the positional relationship of the multiple initial triangular meshes, target triangular meshes located inside the effective virtual entrance pupil region and / or intersecting with the fitted curve of the effective virtual entrance pupil region are selected, and the triangular mesh set is constructed based on the target triangular meshes.

5. A fitting and sampling device for a lens virtual entrance pupil, characterized in that, include: The tracking module is used to generate multiple ray sampling points on an initial plane based on preset lens parameters and an initial field of view, and to perform ray tracing on the multiple ray sampling points to obtain effective ray sampling points; The scanning module is used to systematically scan the effective light sampling points. The systematic scanning is performed in the order of rows and columns, recording the first and last effective light sampling points in each row and each column. The first and last effective light sampling points in each row and each column identified during the scanning process are marked as the initial sampling points of the semi-critical light, and the exploration direction of the semi-critical light is determined. The calculation module is used to calculate the initial point orientation offset of the semi-critical ray based on the semi-critical ray and its exploration direction. The calculation of the initial point orientation offset includes: calculating the distance error to the edge of each optical surface. , ,in, For the first Net aperture of the optical surface For light and the first Intersection of optical surfaces Coordinates, calculate the gradient of the error phase offset The initial point offset is calculated using a first-order Taylor expansion. Select the initial point offset Minimum surface As a target limiting surface; Repeatedly perform ray tracing to the surface The orientation offset is calculated using Newton's iterative formula, which is: ； in, This is the optimal value after iterating the directional offset. For value function, The virtual entrance pupil boundary point corresponding to the semi-critical ray is determined by the first derivative of the value function with respect to the orientation offset. The determination module is used to fit the virtual entrance pupil boundary points of the semi-critical ray with radial basis functions to obtain an effective virtual entrance pupil region, filter the effective virtual entrance pupil region to determine a triangular mesh set, and perform ray tracing based on the triangular mesh set.

6. The apparatus according to claim 5, characterized in that, The tracking module is specifically used for: Determine the coordinates of each light sampling point; Based on the coordinates of each ray sampling point and the initial field of view, the initial direction of each ray sampling point is determined, and the ray equation is constructed according to the initial direction of each ray sampling point; Based on the preset surface equation and the light equation, the intersection points of the light rays generated by each light sampling point according to the corresponding initial direction with multiple lens surfaces are calculated to obtain multiple intersection points; Based on the preset Snell's law, the refraction direction of the light ray corresponding to each light sampling point at the corresponding intersection point is calculated, and the effective light ray reaching the preset image plane after refraction is determined. The intersection point of the effective light ray with the multiple lens surfaces is taken as the effective light sampling point.

7. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the lens virtual entrance pupil fitting and sampling method as described in any one of claims 1-4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the lens virtual entrance pupil fitting and sampling method as described in any one of claims 1-4.

9. A computer program product, said computer program product storing a computer program, characterized in that, When the program is executed by the processor, it implements the fitting and sampling method for the virtual entrance pupil of the lens as described in any one of claims 1-4.

Images