Mirror surface position and shape combined optimization method based on auxiliary plane mirror orthogonal constraint

CN122906156APending Publication Date: 2026-10-13CHANGCHUN INST OF OPTICS FINE MECHANICS & PHYSICS CHINESE ACAD OF SCI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611383907.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-08
Publication Date
2026-10-13

AI Technical Summary

Technical Problem

[0005]然而,现有正交约束方法主要面向相机位姿估计或平面镜几何求解,并未与反射式PMD系统中的显示屏位姿初始化、待测镜初始位姿估计以及后续面形-位姿联合优化过程形成统一技术方案

Benefits of technology

本发明创造所述的基于辅助平面镜正交约束的镜面面形位姿联合优化方法,利用辅助平面镜反射形成的镜像点正交约束为PMD系统几何初始化提供稳定约束,提高显示屏位姿标定和辅助平面镜几何参数求解的可靠性;在面形反演前估计待测镜初始位姿,使后续非线性优化从合理几何邻域开始,降低错误收敛风险;将镜像点正交约束初始化与自动微分反射光线追迹模型相结合,实现待测镜面形残差系数、位姿修正参数以及可选曲率参数的统一优化;降低待测镜初始位姿误差与低阶面形项之间的耦合,提高平面镜、球面镜、非球面镜及其他镜面反射光学元件检测的稳定性;有助于避免仅依赖屏幕点残差最小化时出现的残差较小但面形和位姿分离错误的问题;通过稳定的系统几何初始化和面形位姿联合优化,提高反射式PMD面形检测过程中的全局几何一致性和优化收敛稳定性。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122906156A_ABST
    Figure CN122906156A_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of optical detection, and more particularly to a mirror surface shape and position joint optimization method based on auxiliary plane mirror orthogonal constraint, which uses binocular cameras and a display screen to perform phase deflection measurement on multiple groups of auxiliary plane mirrors; establishes geometric constraints based on reflective imaging, and determines the position of the display screen based on the orthogonal constraint of the intersection line of the constructed mirror image point connection line and the plane mirror; performs deflection observation on the to-be-measured mirror to determine the initial position and introduce a position correction parameter, uses a differentiable reflective ray tracing model to calculate a predicted screen point, and according to the observation residual between the predicted screen point and the corresponding measured screen point, jointly optimizes the to-be-measured mirror surface shape residual coefficient and the position correction parameter; through iterative updating of the to-be-measured mirror surface shape parameter and the position correction parameter, the final position of the to-be-measured mirror and the surface shape detection result are obtained. By introducing the orthogonal constraint with clear geometric meaning and performing joint optimization, the global geometric consistency and optimization convergence stability of the phase deflection measurement are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of optical detection technology, and particularly relates to a joint optimization method for mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors. Background Technology

[0002] In a phase-measuring deflectometry (PMD) system, the spatial geometric relationship between the camera, display screen, and the mirror under test directly determines the mapping relationship between camera pixels, mirror surface points, and display screen points. There is significant coupling between the display screen pose, the initial pose of the mirror under test, and the surface shape parameters of the mirror under test. This is especially true in the testing of curved reflective elements such as spherical and aspherical mirrors, where minute pose errors may be incorrectly absorbed into lower-order surface shape or curvature terms.

[0003] Traditional PMD systems typically rely on camera calibration, display calibration, and the mechanical placement of the mirror under test (DUT) to determine the initial geometric relationships. In actual measurements, display pose errors can cause systematic deviations between measured and predicted screen points. The initial pose error of the DUT can be coupled with low-order surface shape terms such as tilt, defocus, and astigmatism. If the initial geometry is unreliable, subsequent nonlinear optimization can easily enter an incorrect convergence neighborhood, resulting in small screen point residuals but a failure to correctly separate the surface shape and pose.

[0004] Based on the existing orthogonal constraint concept of plane mirror reflection geometry, that is, the line connecting the mirror images of the same point in space with respect to the two plane mirrors and the intersection line of the two plane mirrors satisfy the orthogonal relationship, this constraint can provide stable geometric information for the estimation of camera pose related to the mirror surface.

[0005] However, existing orthogonal constraint methods are mainly aimed at camera pose estimation or plane mirror geometry solution, and have not formed a unified technical solution with the display pose initialization, the initial pose estimation of the mirror under test, and the subsequent surface shape-pose joint optimization process in reflective PMD systems. Summary of the Invention

[0006] In view of this, the present invention aims to provide a joint optimization method for mirror surface shape and pose based on orthogonal constraints of an auxiliary plane mirror. By introducing orthogonal and geometric constraints with clear geometric meaning through an auxiliary plane mirror, the optimal pose of the display screen and the initial pose of the mirror under test are determined. Then, the residual surface shape and pose correction amount of the mirror under test are jointly optimized to improve the global geometric consistency and optimization convergence stability in the surface shape detection process of reflective phase deflection measurement.

[0007] To achieve the above objectives, the technical solution created by this invention is implemented as follows: A joint optimization method for mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors includes: S1: Phase deflection measurement is performed by observing multiple sets of effective and independent auxiliary plane mirrors using a binocular camera and a display screen; S2: Based on the reflection imaging relationship of the auxiliary plane mirrors, establish the geometric constraints between the spatial reference point in the display screen and its mirror image points formed by different auxiliary plane mirrors; S3: Construct an orthogonal constraint between the line connecting the mirror points of any two auxiliary plane mirrors at the same spatial reference point and the intersection line between the two current auxiliary plane mirrors; S4: Construct a geometric optimization model based on geometric and orthogonal constraints; obtain the optimal pose of the display screen through the geometric optimization model; S5: Based on the optimal pose of the display screen, perform phase deflection observation on the mirror under test, and determine the initial pose of the mirror under test based on the observation results; based on the pose of the display screen and the initial pose of the mirror under test, introduce the pose correction parameters of the mirror under test. S6: Calculate the predicted screen points corresponding to the effective pixels of the binocular camera on the display screen using a differentiable reflective ray tracing model, and jointly optimize the residual coefficients of the mirror shape under test and the pose correction parameters based on the observation residuals between the predicted screen points and the corresponding measured screen points. S7: Update the surface shape and current pose of the mirror under test with the surface shape residual coefficient and pose correction parameters optimized in step S6, and repeat step S6 until the preset convergence conditions of the surface shape residual coefficient and pose correction parameters are met, and obtain the final pose and surface shape detection results of the mirror under test.

[0008] Furthermore, step S1 also includes: converting the reference point on the display screen into spatial coordinates in the world coordinate system using the following formula to obtain the spatial reference point: ; in, This represents the coordinates of the i-th reference point on the display screen in the local coordinate system of the display screen. Indicates reference point The corresponding spatial reference point in the world coordinate system, R WS and T WS These represent the rotation matrix and translation vector of the display screen relative to the world coordinate system, respectively.

[0009] Furthermore, step S2 includes: The theoretical mirror point of the spatial reference point on each auxiliary plane mirror is determined by the following formula: ; in, This represents the theoretical mirror image of the i-th spatial reference point on the display screen, formed by the p-th auxiliary plane mirror. Let n represent the i-th spatial reference point on the display screen. pLet P represent the unit normal vector of the surface of the p-th auxiliary plane mirror. p This represents a point on the surface of the p-th auxiliary plane mirror; The acquired images are reconstructed using binocular observation or screen correspondence to obtain the observed mirror points corresponding to the theoretical mirror points; Calculate the consistency residual between the theoretical mirror point and the observed mirror point. Based on the consistency residual, the geometric constraint of the following formula is obtained: ; Among them, E mirror Represents geometric constraints, This represents the geometric constraint weights of the corresponding mirror point. This indicates a consistent residual.

[0010] Furthermore, the orthogonal constraint in step S3 is: ; Among them, E orth This indicates an orthogonal constraint relationship. and Let represent the theoretical mirror image points formed by the i-th spatial reference point on the display screen through the p-th and q-th auxiliary plane mirrors, respectively, and let Ω represent the set of auxiliary plane mirrors used to construct position orthogonal constraints. The direction m of the intersection line between the two auxiliary plane mirrors is indicated. pq The direction of the unit intersection line.

[0011] Furthermore, the geometric optimization model in step S4 is as follows: ; Among them, E mirror E represents geometric constraints. orth E represents orthogonal constraint. rep Denotes the binocular reprojection constraint, λ orth and λ rep θ represents the weight coefficient of the corresponding constraint. S The pose parameter θ represents the position of the display screen. A Indicates the geometric parameters of the auxiliary plane mirror. and Let represent the optimized display pose parameters and auxiliary plane mirror geometric parameters, respectively; where the binocular reprojection constraint is: ; in, This represents the predicted projection position of the theoretical mirror point of the i-th spatial reference point corresponding to the p-th auxiliary plane mirror onto the image plane of the c-th camera, using a pre-calibrated projection model of the c-th camera. This represents the corresponding actual observed pixel. This represents the corresponding reprojection constraint weight.

[0012] Furthermore, step S5 includes: S51: Acquire images of the mirror under test using a binocular camera, extract the effective area of ​​the mirror under test from each camera image, and calculate the geometric center of the effective area; S52: Select multiple preset screen feature points at the center of the display screen and its neighborhood, and load a reference pattern containing the screen feature points onto the display screen. S53: The reference pattern in step S52 is captured by the binocular camera after being reflected by the mirror under test. The phase solution and screen coordinate mapping are performed on the acquired image to obtain the binocular observation point corresponding to each screen feature point in step S52 in the acquired image. S54: The surface shape of the mirror under test is represented by the superposition of the reference surface and the residual surface shape basis functions. The parameters of the reference surface in the mirror under test are set to constant values, the residual surface shape coefficient of the mirror under test is temporarily set to zero, and the six-degree-of-freedom rigid body pose of the mirror under test relative to the world coordinate system is used as the parameter to be estimated. S55: Randomly generate multiple candidate initial poses for the mirror under test. Based on the binocular camera parameters, the optimal pose of the display screen, and the reference surface of the mirror under test in step S54, calculate the predicted screen point corresponding to the binocular observation point in each step S53 using the forward tracing model of reflected light. S56: Compare the predicted screen points obtained in step S55 with the corresponding screen feature points in step S52, and combine the consistency between the projection position of the spatial center of the mirror under test in the binocular image and the image center of the effective area in the mirror under test to construct the initial pose optimization target of the mirror under test. S57: The initial pose of the mirror under test is obtained by minimizing the initial pose optimization target in step S56. S58: Based on the initial pose of the mirror under test obtained in step S57, construct pose correction parameters to describe the translation and rotation corrections of the actual pose of the mirror under test relative to the initial pose. The relationship between the initial pose of the mirror under test, the pose correction parameters, and the current pose of the mirror under test is as follows: ; Among them, A WM ξ represents the current pose of the mirror under test. M This represents the pose correction parameters used to describe the translation and rotation corrections of the actual pose of the mirror under test relative to its initial pose. This represents the rigid body transformation matrix composed of pose correction parameters. This indicates the initial pose of the mirror to be tested.

[0013] Furthermore, the initial pose optimization objective in step S56 is: ; Among them, E init Let K represent the initial pose optimization objective, and s represent the total number of screen feature points. k Let λ represent the k-th screen feature point. cen Represents the weighting coefficient, π c t represents the projection function from a 3D spatial point established based on the pre-calibrated intrinsic and extrinsic parameters of the c-th camera onto the image plane of that camera. WM This represents the position vector of the origin of the local coordinate system where the mirror under test is located in the world coordinate system.

[0014] Furthermore, step S6 includes: S61: Using the reflected ray tracing model, based on the current pose and surface parameters of the mirror under test, as well as the optimal pose of the display screen, the predicted screen points corresponding to the effective pixels in the current binocular camera's captured image are obtained on the display screen. S62: Using the observation residual between the predicted screen point obtained in step S61 and the corresponding measured screen point obtained from observation as the objective, construct a joint optimization function; S63: Optimize the surface residual coefficients and pose correction parameters of the mirror under test using the joint optimization function constructed in step S62.

[0015] Furthermore, the ray tracing model in step S61 is as follows: ; in, Let represent the predicted screen point corresponding to the j-th valid pixel in the image acquired by the c-th camera. This represents the forward operator for tracing differentiable reflective rays. This represents the j-th valid pixel in the image acquired by the c-th camera. Let θ represent the parameters of the c-th camera. S The display shows the pose parameters, 'a' represents the residual surface shape coefficient of the mirror under test, and R... M This represents the curvature parameter of the reference surface of the mirror under test.

[0016] Furthermore, the joint optimization function in step S62 is: ; in, Represents the observation weight matrix. Let j represent the predicted screen point in the image acquired by the c-th camera. λ represents the corresponding measured screen point. a Let L represent the regularization weights of the residual surface shape coefficients, L represent the regularization matrix applied to the surface shape residual coefficients, a represent the residual surface shape coefficients of the residual surface shape basis functions, and λ represent the residual surface shape coefficients. ξThis represents the regularization weights for pose correction.

[0017] Compared with the prior art, the present invention can achieve the following beneficial effects: This invention presents a joint optimization method for mirror surface shape and pose based on orthogonal constraints of an auxiliary plane mirror. It utilizes the orthogonal constraints of mirror points formed by the reflection of the auxiliary plane mirror to provide stable constraints for the geometric initialization of the PMD system, improving the reliability of display screen pose calibration and the solution of auxiliary plane mirror geometric parameters. Before surface shape inversion, the initial pose of the mirror under test is estimated, allowing subsequent nonlinear optimization to start from a reasonable geometric neighborhood, reducing the risk of erroneous convergence. The orthogonal constraint initialization of mirror points is combined with an automatic differential reflection ray tracing model to achieve unified optimization of the residual coefficients, pose correction parameters, and optional curvature parameters of the mirror surface shape. This reduces the coupling between the initial pose error of the mirror under test and low-order surface shape terms, improving the stability of detection for plane mirrors, spherical mirrors, aspherical mirrors, and other specular reflective optical elements. It helps avoid the problem of small residuals but incorrect separation of surface shape and pose when relying solely on minimizing screen point residuals. Through stable system geometric initialization and joint optimization of surface shape and pose, it improves the global geometric consistency and optimization convergence stability in the reflective PMD surface shape detection process. Attached Figure Description

[0018] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 A flowchart illustrating the joint optimization method for mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors as described in an embodiment of the present invention; Figure 2 A schematic diagram of the first stage of the invention as described in the embodiments of the present invention; Figure 3 A schematic diagram of the PMD system composed of a binocular camera, a display screen, and an auxiliary plane mirror as described in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the orthogonal constraint between the line connecting the mirror points formed by different auxiliary plane mirrors of the same spatial reference point in an embodiment of the present invention and the intersection line of the corresponding auxiliary plane mirror; Figure 5 This is a schematic diagram of the joint optimization process of the mirror surface shape and pose described in an embodiment of the present invention.

[0019] Explanation of reference numerals in the attached figures: 1. Display screen; 2. Binocular camera; 3. Auxiliary plane mirror. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0021] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0022] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] like Figure 1 As shown in the embodiment of the present invention, the joint optimization method for mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors includes two stages. The first stage is the geometric initialization of the PMD system based on the orthogonal constraints of the mirror image points of the auxiliary plane mirror, which mainly includes the acquisition of reflection observations of the auxiliary plane mirror, extraction of mirror image points, establishment of mirror image point constraints, establishment of orthogonal constraints of mirror image points, and solving for the display pose and geometric parameters of the auxiliary plane mirror. The second stage is the joint optimization of surface shape and pose based on the initialized geometric parameters, which mainly includes the initial pose estimation of the mirror under test and the joint optimization of the surface shape and pose of the mirror under test.

[0024] Specifically, in combination Figure 1 and Figure 2 The first stage of the joint optimization method for mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors provided by this invention includes: S1: Phase deflection measurement is performed by observing multiple sets of effective and independent auxiliary plane mirrors using a binocular camera and a display screen.

[0025] In this embodiment of the invention, the PMD system composed of a binocular camera, a display screen, and an auxiliary plane mirror is as follows: Figure 3As shown, the display screen 1 projects a reference pattern or phase-shifted fringes, and the binocular camera 2 acquires the image reflected by the auxiliary plane mirror 3. Furthermore, three auxiliary plane mirrors 3 are specifically configured, and each of the three auxiliary plane mirrors 3 interacts with the binocular camera 2 to perform effective and independent mirror reflection observations. This is used to form different mirror images of the spatial reference point on the display screen 1 under different orientations of the auxiliary plane mirrors 3. "Effective and independent" means that the auxiliary plane mirrors 3 in different orientations can provide sufficient geometric constraints so that the intersection direction and normal vector of the auxiliary plane mirrors 3 can be stably solved. In other embodiments, a single auxiliary plane mirror 3 can be used, i.e., the same auxiliary plane mirror 3 can be set in multiple different spatial orientations, allowing the binocular camera 2 to cooperate with the auxiliary plane mirrors 3 in different spatial orientations to complete multiple sets of effective and independent mirror reflection observations.

[0026] During the above-mentioned process of obtaining reflection observations using auxiliary plane mirrors, the binocular camera and the display screen remain fixed. Multiple sets of effective and independent reflection observations are obtained by setting up multiple auxiliary plane mirrors with different orientations or changing the orientation of the same auxiliary plane mirror.

[0027] In some embodiments, step S1 further includes: Let the i-th reference point in the local coordinate system of the display screen be... The spatial coordinates of the reference point in the world coordinate system are determined by the following formula, thus obtaining the spatial reference point: ; in, Indicates the reference point on the display screen The corresponding spatial reference point in the world coordinate system, R WS and T WS These represent the rotation matrix and translation vector of the display screen relative to the world coordinate system, respectively.

[0028] S2: Based on the reflection imaging relationship of the auxiliary plane mirrors, establish geometric constraints between the spatial reference point in the display screen and its mirror points formed by different auxiliary plane mirrors, ensuring consistency between the mirror points.

[0029] The plane equation of the p-th auxiliary plane mirror is: n p T (XP p )=0; Where, n p Let P represent the unit normal vector of the surface of the p-th auxiliary plane mirror. pLet X represent a point on the surface of the p-th auxiliary plane mirror, and let X represent the coordinates of any point on the surface of the auxiliary plane mirror. Based on the plane equation of the auxiliary plane mirror, the theoretical mirror image point formed by any spatial reference point in the display screen through the auxiliary plane mirror can be determined, and the mirror geometry relationship between the spatial reference point and the theoretical mirror image point can be established.

[0030] In some embodiments, based on the above reflection imaging relationship of the auxiliary plane mirror, step S2 includes: The theoretical mirror point of the spatial reference point on each auxiliary plane mirror is determined by the following formula: ; in, This represents the theoretical mirror image of the i-th spatial reference point in the display screen after being reflected by the p-th auxiliary plane mirror. This formula describes the mirror geometry of the spatial reference point in the display screen after being reflected by the auxiliary plane mirror. The acquired images are reconstructed using binocular observation or screen correspondence to obtain the observed mirror points corresponding to the theoretical mirror points; The consistency residual between the theoretical mirror point and the observed mirror point is calculated using the following formula in this embodiment of the invention: ; in, Indicates consistent residuals, Represents the theoretical mirror point The corresponding observation mirror point; Based on the consistency residuals, the geometric constraints are obtained as follows: ; Among them, E mirror Represents geometric constraints, This represents the geometric constraint weight of the corresponding mirror point.

[0031] In this embodiment of the invention, the process of recovering the observed mirror point from the acquired image using binocular observation or screen correspondence is derived from "RI Hartley and P. Sturm, “Triangulation,” Computer Vision and Image Understanding, vol. 68, no. 2, pp. 146–157, 1997, DOI: 10.1006 / cviu.1997.0547". The process of obtaining the observed mirror point includes: for the p-th auxiliary plane mirror and the i-th reference point on the display screen, through phase calculation and screen coordinate mapping, determining the reflection pixels corresponding to the same display screen reference point in the two camera images respectively; according to the pre-calibrated binocular camera intrinsic and extrinsic parameters, back-projecting the two reflection pixels into back-projection rays using the following formula: ; Among them, C c Let c represent the optical center of the c-th camera, where c = {1, 2}. This indicates the unit direction of the corresponding back-projected ray. Let represent the corresponding back-projection ray, and τ represent the scale parameter along the direction of the back-projection ray. Since two spatial rays generally do not intersect strictly under noise conditions, the three-dimensional observation mirror point corresponding to the reference point can be obtained by least-squares trigonometric measurement, as shown in the following equation: ; Where Q represents the three-dimensional observation mirror point to be determined.

[0032] S3: Construct the orthogonal constraint between the line connecting the mirror points of any two auxiliary plane mirrors of the same spatial reference point in step S2 and the intersection line between the two current auxiliary plane mirrors.

[0033] like Figure 4 As shown, for the p-th and q-th auxiliary plane mirrors, their unit normal vectors are n and n respectively. p and n q The direction of the intersection of the two auxiliary plane mirrors is m. pq =n p ×n q For the same spatial reference point P i The theoretical mirror images formed by reflection from the p-th and q-th auxiliary plane mirrors are respectively and According to mirror geometry, the line connecting two theoretical mirror points and the intersection line of the two auxiliary plane mirrors must be orthogonal. For an effective combination of non-parallel auxiliary mirrors, ||n| must be orthogonal. p ×n q ||2>ε, where ε represents a preset positive threshold used to determine whether the two auxiliary plane mirrors form a valid non-parallel combination and to exclude geometric degradation caused by near-parallelism. Therefore, orthogonal residuals of the mirror points can be constructed. : ; in, The direction m of the intersection line between the two auxiliary plane mirrors is indicated. pq The direction of the unit intersection line, when the geometric parameters satisfy the ideal orthogonality constraint, has: .

[0034] Based on the above-mentioned orthogonal residuals of the mirror points, orthogonal constraint relationships are constructed between each auxiliary plane mirror. Specifically, in some embodiments, the orthogonal constraints are as follows: ; Among them, E orthThis indicates an orthogonal constraint relationship. and Let represent the theoretical mirror images formed by the i-th spatial reference point on the display screen through the p-th and q-th auxiliary plane mirrors, respectively, and Ω represent the set of auxiliary plane mirrors used to construct the position orthogonality constraint. This constraint utilizes the relationship between the mirror images formed by reflections from different auxiliary plane mirrors to provide additional stability constraints for the subsequent geometric initialization of the PMD system.

[0035] S4: Construct a geometric optimization model based on geometric constraints and orthogonal constraints; obtain the optimal pose of the display screen through the geometric optimization model.

[0036] In some embodiments, the constructed geometric optimization model is: ; Among them, E mirror E represents geometric constraints. orth E represents orthogonal constraint. rep λ represents the binocular reprojection constraint between the predicted projection position of the theoretical mirror point formed by the spatial reference point on the display screen via the auxiliary plane mirror and the actual observed pixel of the theoretical mirror point in the binocular camera. orth and λ rep θ represents the weight coefficient of the corresponding constraint. S Represents the set of pose parameters of the display screen, θ A This represents the set of geometric parameters of the auxiliary plane mirror. and These represent the optimized display screen pose parameters and auxiliary plane mirror geometric parameters, respectively. The binocular reprojection constraint is as follows: ; in, This represents the predicted projection position of the theoretical mirror point of the i-th spatial reference point corresponding to the p-th auxiliary plane mirror onto the image plane of the c-th camera, using a pre-calibrated projection model of the c-th camera. This represents the corresponding actual observed pixel. This represents the corresponding reprojection constraint weight.

[0037] By solving the above optimization problem, the pose parameters of the display screen relative to the world coordinate system or the binocular camera system, as well as the geometric parameters of the auxiliary plane mirror, are obtained. The initialized display screen pose parameters are used as fixed system geometric parameters in the subsequent inversion of the surface shape of the mirror under test, thereby reducing the impact of the display screen's geometric error on the surface shape recovery result.

[0038] The second stage of the joint optimization method for mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors provided by this invention includes: S5: Based on the optimal pose of the display screen in step S4, perform phase deflection observation on the mirror under test, and determine the initial pose of the mirror under test based on the observation results; based on the pose of the display screen and the initial pose of the mirror under test, introduce the pose correction parameters of the mirror under test.

[0039] In some embodiments, step S5 includes: S51: Acquire images of the mirror under test using a binocular camera, extract the effective area of ​​the mirror under test from each camera image, and calculate the geometric center of the effective area.

[0040] The geometric center is used to provide initial position constraints for the spatial center of the mirror under test in the binocular image. In this embodiment of the invention, the geometric center is obtained by the following formula: ; in, Indicates the geometric center of the effective region. This represents the valid region, where μ represents the pixel coordinates within the valid region. S52: Select multiple preset screen feature points at the center of the display screen and its neighborhood, and load a reference pattern containing the screen feature points onto the display screen.

[0041] S53: The reference pattern in step S52 is captured by the binocular camera after being reflected by the mirror under test. The acquired image is subjected to phase calculation and screen coordinate mapping to obtain the binocular observation point corresponding to each screen feature point in step S52 in the acquired image.

[0042] S54: The surface shape of the mirror under test is represented by the superposition of the reference surface and the residual surface shape basis functions. The parameters of the reference surface in the mirror under test are set to constant values, the residual surface shape coefficient of the mirror under test is temporarily set to zero, and the six-degree-of-freedom rigid body pose of the mirror under test relative to the world coordinate system is used as the parameter to be estimated.

[0043] The surface shape of the mirror to be tested in this embodiment of the invention is specifically represented as follows: ; Among them, z M Let x represent the surface shape function of the mirror under test, (x) M ,y M Let f0 represent any point on the surface of the mirror under test, f0 represent the reference plane of the mirror under test, α0 represent the fixed parameters of the reference plane, and R M B represents the radius of curvature parameter of the reference surface. g Let a denote the residual surface basis function of the g-th term. g Let G represent the residual surface shape coefficient of the g-th term, and G represent the total number of terms in the residual surface shape basis functions. S55: Randomly generate multiple candidate initial poses for the mirror under test. Based on the binocular camera parameters, the optimal pose of the display screen, and the reference surface of the mirror under test in step S54, calculate the predicted screen point corresponding to the binocular observation point in each step S53 using the forward tracing model of reflected light.

[0044] In this embodiment of the invention, it can be represented by the following formula: ; in, Let θ represent the observation point in the c-th camera corresponding to the k-th screen feature point. c Indicates the parameters of a stereo camera, A WM Π0 represents the candidate initial pose of the mirror under test, and Π0 represents the forward tracing model of the reflected ray.

[0045] S56: Compare the predicted screen points obtained in step S55 with the corresponding screen feature points in step S52, and combine the consistency between the projection position of the spatial center of the mirror under test in the binocular image and the image center of the effective area in the mirror under test to construct the initial pose optimization target of the mirror under test.

[0046] In some embodiments, the initial pose optimization objective of the mirror under test is: ; Among them, E init Let K represent the initial pose optimization objective, and s represent the total number of screen feature points. k Let λ represent the k-th screen feature point. cen Represents the weighting coefficient, π c t represents the projection function from a 3D spatial point established based on the pre-calibrated intrinsic and extrinsic parameters of the c-th camera onto the image plane of that camera. WM This represents the position vector of the origin of the local coordinate system of the mirror under test in the world coordinate system, i.e., the translation component of the current candidate pose of the mirror under test. In this embodiment of the invention, the origin of the local coordinate system is used to characterize the spatial center of the mirror under test. The first term in the initial pose optimization objective is the reflection observation residual of screen feature points, used to constrain the consistency between the predicted screen points calculated under the candidate pose of the mirror under test and the corresponding known screen feature points; the second term in the initial pose optimization objective is the soft constraint of the center position of the mirror under test, used to constrain the consistency between the predicted projection position of the local coordinate origin of the mirror under test in the binocular image and the center of the effective area image of the mirror under test, with a weight coefficient λ. cen This refers to the weighting coefficient of the center position constraint term.

[0047] S57: The initial pose of the mirror under test is obtained by minimizing the initial pose optimization target in step S56.

[0048] In this embodiment of the invention, step S57 can be represented by the following formula: .

[0049] in, This indicates the initial pose of the mirror under test.

[0050] In this embodiment of the invention, the initial pose of the mirror under test can be expressed as: ; in, This indicates the initial pose of the mirror under test. and Let denot represent the initial rotation matrix and initial translation vector of the local coordinate system of the mirror under test relative to the world coordinate system, respectively. The initial rotation matrix is ​​used to characterize the initial position of the spatial center of the mirror under test in the world coordinate system. Used to determine the initial orientation of the local coordinate system of the mirror under test, where the Z-axis corresponds to the initial axial direction or initial normal of the mirror under test.

[0051] The initial pose is used to provide a reasonable initial state for subsequent joint optimization of surface pose. The purpose of this process is not to directly output the final surface shape result, but to provide a stable initial geometric state for subsequent reflection ray tracing optimization, thereby reducing the risk of subsequent optimization getting trapped in the wrong convergence neighborhood.

[0052] S58: Based on the initial pose of the mirror under test obtained in step S57, construct pose correction parameters to describe the translation and rotation corrections of the actual pose of the mirror under test relative to the initial pose.

[0053] In step S58, the relationship between the initial pose of the mirror under test, the pose correction parameters, and the current pose of the mirror under test is as follows: ; Among them, A WM ξ represents the current pose of the mirror under test. M Indicates pose correction parameters. This represents the rigid body transformation matrix composed of pose correction parameters.

[0054] After obtaining the initial pose of the mirror under test, according to Figure 5 The execution steps S6 to S7 of the joint optimization process for surface shape and pose are shown.

[0055] S6: Calculate the predicted screen points corresponding to the effective pixels of the binocular camera on the display screen using a differentiable reflective ray tracing model, and jointly optimize the residual coefficients of the mirror shape under test and the pose correction parameters based on the observation residuals between the predicted screen points and the corresponding measured screen points.

[0056] In some embodiments, step S6 includes: S61: Using the reflected ray tracing model, based on the current pose and surface parameters of the mirror under test, as well as the optimal pose of the display screen, the predicted screen points corresponding to the effective pixels in the current binocular camera's captured image are obtained on the display screen.

[0057] In some embodiments, the reflected ray tracing model is established as follows: ; in, Let represent the predicted screen point corresponding to the j-th valid pixel in the image acquired by the c-th camera. This represents the forward operator for tracing differentiable reflective rays. This represents the j-th valid pixel in the image acquired by the c-th camera. Let R represent the parameters of the c-th camera in a stereo camera setup, a represent the residual surface shape coefficients of the residual surface shape basis functions, and R... M This represents the curvature parameter of the reference surface of the mirror under test.

[0058] In this embodiment of the invention, the basic ideas of automatic differentiation and parameter differentiation based on differentiable ray propagation (i.e., the forward operator for differentiable reflective ray tracing) are existing technologies. For example, relevant research by Wang et al. (C. Wang, N. Chen, and W. Heidrich, “Towards self-calibrated lens metrology by differentiablerefractive deflectometry,” Optics Express, vol. 29, pp. 30284–30295, 2021.) should be referenced. It should be noted that this invention does not take the automatic differentiation method itself as an innovation point, but rather applies this existing technology to establish a forward model for differentiable reflective ray tracing of effective pixels of the camera reflected by the mirror under test to the display screen for reflective phase deflection measurement. This allows for subsequent joint optimization of the surface shape parameters and pose correction parameters of the mirror under test driven by the observation residual between the predicted screen point and the measured screen point.

[0059] S62: Using the observation residual between the predicted screen point obtained in step S61 and the corresponding measured screen point obtained from observation as the objective, construct a joint optimization function.

[0060] In some embodiments, the joint optimization function is: ; in, Represents the observation weight matrix. λ represents the corresponding measured screen point. a λ represents the regularization weight of the residual surface coefficients, L represents the regularization matrix applied to the surface residual coefficients, and λ represents the regularization weight.ξ This represents the pose correction regularization weight.

[0061] S63: Using the joint optimization function constructed in step S62, the surface shape residual coefficients and pose correction parameters of the mirror under test are optimized to obtain the optimized surface shape residual coefficients and pose correction parameters of the mirror under test. In this embodiment of the invention, when curvature estimation is required, the curvature parameters are optimized simultaneously.

[0062] S7: Update the surface shape and current pose of the mirror under test with the surface shape residual coefficient and pose correction parameters optimized in step S6, and repeat step S6 until the preset convergence conditions of the surface shape residual coefficient and pose correction parameters are met, and obtain the final pose and surface shape detection results of the mirror under test.

[0063] Specifically, using the joint optimization function in step S6, step S6 is repeated to jointly optimize the surface residual coefficients and pose correction parameters of the mirror under test; based on the obtained final surface residual coefficients and pose correction parameters of the mirror under test, the final pose of the mirror under test and the surface detection results are determined.

[0064] Specifically, the final pose of the mirror under test As shown in the following formula: ; in, This represents the pose correction parameters obtained from the joint optimization.

[0065] Final surface shape detection results As shown in the following formula: ; in, denoted as the g-th residual surface shape coefficient obtained from the joint optimization. Let G represent the g-th residual surface basis function, and let G represent the total number of residual surface basis functions.

[0066] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0067] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for joint optimization of mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors, characterized in that, include: S1: Phase deflection measurement is performed by observing multiple sets of effective and independent auxiliary plane mirrors using a binocular camera and a display screen; S2: Based on the reflection imaging relationship of the auxiliary plane mirrors, establish the geometric constraints between the spatial reference point in the display screen and its mirror image points formed by different auxiliary plane mirrors; S3: Construct an orthogonal constraint between the line connecting the mirror points of any two auxiliary plane mirrors at the same spatial reference point and the line of intersection between the two current auxiliary plane mirrors; S4: Construct a geometric optimization model based on geometric and orthogonal constraints; obtain the optimal pose of the display screen through the geometric optimization model; S5: Based on the optimal pose of the display screen, perform phase deflection observation on the mirror under test, and determine the initial pose of the mirror under test based on the observation results; based on the pose of the display screen and the initial pose of the mirror under test, introduce the pose correction parameters of the mirror under test. S6: Calculate the predicted screen points corresponding to the effective pixels of the binocular camera on the display screen using a differentiable reflective ray tracing model, and jointly optimize the residual coefficients of the mirror shape under test and the pose correction parameters based on the observation residuals between the predicted screen points and the corresponding measured screen points. S7: Update the surface shape and current pose of the mirror under test with the surface shape residual coefficient and pose correction parameters optimized in step S6, and repeat step S6 until the preset convergence conditions of the surface shape residual coefficient and pose correction parameters are met, and obtain the final pose and surface shape detection results of the mirror under test.

2. The method for joint optimization of mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors according to claim 1, characterized in that, Step S1 also includes: The spatial reference point is obtained by converting the reference point on the display screen to spatial coordinates in the world coordinate system using the following formula: ; in, This represents the coordinates of the i-th reference point on the display screen in the local coordinate system of the display screen. Indicates reference point The corresponding spatial reference point in the world coordinate system, R WS and T WS These represent the rotation matrix and translation vector of the display screen relative to the world coordinate system, respectively.

3. The method for joint optimization of mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors according to claim 1, characterized in that, Step S2 includes: The theoretical mirror point of the spatial reference point on each auxiliary plane mirror is determined by the following formula: ; in, This represents the theoretical mirror image of the i-th spatial reference point on the display screen, formed by the p-th auxiliary plane mirror. Let n represent the i-th spatial reference point on the display screen. p Let P represent the unit normal vector of the surface of the p-th auxiliary plane mirror. p This represents a point on the surface of the p-th auxiliary plane mirror; The acquired images are reconstructed using binocular observation or screen correspondence to obtain the observed mirror points corresponding to the theoretical mirror points; Calculate the consistency residual between the theoretical mirror point and the observed mirror point. Based on the consistency residual, the geometric constraint of the following formula is obtained: ; Among them, E mirror Represents geometric constraints, This represents the geometric constraint weights of the corresponding mirror point. This indicates a consistent residual.

4. The method for joint optimization of mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors according to claim 1, characterized in that, The orthogonal constraint in step S3 is: ; Among them, E orth This indicates an orthogonal constraint relationship. and Let represent the theoretical mirror image points formed by the i-th spatial reference point on the display screen through the p-th and q-th auxiliary plane mirrors, respectively, and let Ω represent the set of auxiliary plane mirrors used to construct position orthogonal constraints. The direction m of the intersection line between the two auxiliary plane mirrors is indicated. pq The direction of the unit intersection line.

5. The method for joint optimization of mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors according to claim 1, characterized in that, The geometric optimization model in step S4 is as follows: ; Among them, E mirror E represents geometric constraints. orth E represents orthogonal constraint. rep Denotes the binocular reprojection constraint, λ orth and λ rep θ represents the weight coefficient of the corresponding constraint. S The pose parameter θ represents the position of the display screen. A Indicates the geometric parameters of the auxiliary plane mirror. and These represent the optimized display screen pose parameters and auxiliary plane mirror geometric parameters, respectively. The binocular reprojection constraint is as follows: ; in, This represents the predicted projection position of the theoretical mirror point of the i-th spatial reference point corresponding to the p-th auxiliary plane mirror onto the image plane of the c-th camera, using a pre-calibrated projection model of the c-th camera. This represents the corresponding actual observed pixel. This represents the corresponding reprojection constraint weight.

6. The method for joint optimization of mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors according to claim 1, characterized in that, Step S5 includes: S51: Acquire images of the mirror under test using a binocular camera, extract the effective area of ​​the mirror under test from each camera image, and calculate the geometric center of the effective area; S52: Select multiple preset screen feature points at the center of the display screen and its neighborhood, and load a reference pattern containing the screen feature points onto the display screen. S53: The reference pattern in step S52 is captured by the binocular camera after being reflected by the mirror under test. The phase solution and screen coordinate mapping are performed on the acquired image to obtain the binocular observation point corresponding to each screen feature point in step S52 in the acquired image. S54: The surface shape of the mirror under test is represented by the superposition of the reference surface and the residual surface shape basis functions. The parameters of the reference surface in the mirror under test are set to constant values, the residual surface shape coefficient of the mirror under test is temporarily set to zero, and the six-degree-of-freedom rigid body pose of the mirror under test relative to the world coordinate system is used as the parameter to be estimated. S55: Randomly generate multiple candidate initial poses for the mirror under test. Based on the binocular camera parameters, the optimal pose of the display screen, and the reference surface of the mirror under test in step S54, calculate the predicted screen point corresponding to the binocular observation point in each step S53 using the forward tracing model of reflected light. S56: Compare the predicted screen points obtained in step S55 with the corresponding screen feature points in step S52, and combine the consistency between the projection position of the spatial center of the mirror under test in the binocular image and the image center of the effective area in the mirror under test to construct the initial pose optimization target of the mirror under test. S57: The initial pose of the mirror under test is obtained by minimizing the initial pose optimization target in step S56. S58: Based on the initial pose of the mirror under test obtained in step S57, construct pose correction parameters to describe the translation and rotation corrections of the actual pose of the mirror under test relative to the initial pose. The relationship between the initial pose of the mirror under test, the pose correction parameters, and the current pose of the mirror under test is as follows: ; Among them, A WM ξ represents the current pose of the mirror under test. M This represents the pose correction parameters used to describe the translation and rotation corrections of the actual pose of the mirror under test relative to its initial pose. This represents the rigid body transformation matrix composed of pose correction parameters. This indicates the initial pose of the mirror to be tested.

7. The method for joint optimization of mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors according to claim 6, characterized in that, The initial pose optimization objective in step S56 is: ; Among them, E init Let K represent the initial pose optimization objective, and s represent the total number of screen feature points. k Let λ represent the k-th screen feature point. cen Represents the weighting coefficient, π c t represents the projection function from a 3D spatial point established based on the pre-calibrated intrinsic and extrinsic parameters of the c-th camera onto the image plane of that camera. WM This represents the position vector of the origin of the local coordinate system where the mirror under test is located in the world coordinate system.

8. The method for joint optimization of mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors according to claim 6, characterized in that, Step S6 includes: S61: Using the reflected ray tracing model, based on the current pose and surface parameters of the mirror under test, as well as the optimal pose of the display screen, the predicted screen points corresponding to the effective pixels in the current binocular camera's captured image are obtained on the display screen. S62: Using the observation residual between the predicted screen point obtained in step S61 and the corresponding measured screen point obtained from observation as the objective, construct a joint optimization function; S63: Optimize the surface residual coefficients and pose correction parameters of the mirror under test using the joint optimization function constructed in step S62.

9. The method for joint optimization of mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors according to claim 8, characterized in that, The ray tracing model in step S61 is as follows: ; in, Let represent the predicted screen point corresponding to the j-th valid pixel in the image acquired by the c-th camera. This represents the forward operator for tracing differentiable reflective rays. This represents the j-th valid pixel in the image acquired by the c-th camera. Let θ represent the parameters of the c-th camera. S The display shows the pose parameters, 'a' represents the residual surface shape coefficient of the mirror under test, and R... M This represents the curvature parameter of the reference surface of the mirror under test.

10. The method for joint optimization of mirror surface shape and pose based on orthogonal constraints of auxiliary plane mirrors according to claim 8, characterized in that, The joint optimization function in step S62 is: ; in, Represents the observation weight matrix. Let j represent the predicted screen point in the image acquired by the c-th camera. λ represents the corresponding measured screen point. a λ represents the regularization weights of the residual surface shape coefficients, L represents the regularization matrix applied to the surface shape residual coefficients, a represents the residual surface shape coefficients of the residual surface shape basis functions, and λ represents the residual surface shape coefficients. ξ This represents the regularization weights for pose correction.