Deflection system calibration method and device

Abstract

The invention relates to the technical field of structured light three-dimensional imaging system calibration, in particular to a deflection system calibration method and device. The method comprises the following steps: determining a calibration image displayed on the display screen; determining a phase value of each feature point according to the light intensity value, the amplitude and the position coordinate of each feature point; determining an internal reference initial value and an external reference initial value based on the internal reference matrix, the distortion parameter and the rotation and translation matrix of the camera; performing coordinate correction on each feature point by adopting a preset three-dimensional offset vector to obtain a corrected coordinate of each feature point; and determining a re-projection error based on the position coordinate of each feature point and the corrected coordinate of each feature point, and performing joint optimization on the deflection system based on the re-projection error to obtain an optimized internal parameter and an optimized external parameter. The calibration precision of the deflection system can be improved.

Description

Technical Field

[0001] This application relates to the field of calibration technology for structured light three-dimensional imaging systems, and in particular to a calibration method and apparatus for a deflection system. Background Technology

[0002] Deflection is a non-contact structured light imaging technique that uses light reflection to measure the surface topography of mirrors and mirror-like objects. Similar to other structured light 3D imaging methods, deflection requires precise calibration of the imaging system before topography reconstruction. This involves calibrating the camera's intrinsic parameters (intrinsic parameter matrix and distortion parameters) and the geometric extrinsic parameters (rotation and translation matrices) between the camera and the screen. Because deflection relies on the geometric parameters between the devices provided by calibration for object reconstruction, the accuracy of the system calibration directly determines the accuracy of the 3D reconstruction.

[0003] Calibration of a deflection imaging system has certain unique characteristics. To achieve reflective imaging, the camera and display screen are located on the same side, resulting in the display screen being outside the camera's field of view. Therefore, to achieve accurate calibration of the intrinsic and extrinsic parameters of the deflection imaging system, a plane mirror is required. During calibration, the camera and display screen remain relatively stationary, and the plane mirror is placed opposite them to reflect the calibration pattern displayed on the screen. The camera then captures the mirrored pattern of the screen. By moving the plane mirror to obtain mirrored patterns at multiple angles, extracting feature points from the images, and performing linear solutions, the initial parameters for system calibration can be obtained. Subsequently, a bundle adjustment method is used to nonlinearly optimize the initial values ​​to improve calibration accuracy.

[0004] However, in the calibration process described above, the screen is usually assumed to be an ideal two-dimensional plane, ignoring its actual geometric deformation. The error introduced by this deformation is easily misallocated to the intrinsic and extrinsic parameters of other devices during nonlinear optimization, leading to a decrease in system calibration accuracy and consequently adversely affecting the quality of subsequent 3D reconstruction. Summary of the Invention

[0005] To address at least one of the aforementioned problems, embodiments of this application provide a calibration method and apparatus for a deflection system, which can resolve calibration deviations caused by non-ideal geometric deformation of the screen.

[0006] According to one aspect of the embodiments of this application, a calibration method for a deflection system is proposed, applied to a deflection system including a display screen, a plane mirror, and a camera, the method comprising: A calibration image for display on the screen is determined, the calibration image consisting of a circular array and a phase-shifting fringe pattern, the circular array including a plurality of feature points; The light intensity value and amplitude received by the camera for the calibration image are determined, and the phase value of each feature point is determined based on the light intensity value, the amplitude, and the position coordinates of each feature point. Determine the intrinsic parameter matrix of the camera, the distortion parameters of the camera, and the rotation and translation matrix between the camera and the display screen; The initial values ​​of the camera's intrinsic parameters and the initial values ​​of the extrinsic parameters between the camera and the display screen are determined based on the intrinsic parameter matrix, the distortion parameters, and the rotation and translation matrix. The coordinates of each feature point are corrected using a preset three-dimensional offset vector to obtain the corrected coordinates of each feature point. The reprojection error is determined based on the position coordinates of each feature point and the corrected coordinates of each feature point. The deflection system is then jointly optimized based on the reprojection error to obtain the optimized intrinsic parameters of the camera and the optimized extrinsic parameters between the camera and the display screen.

[0007] In the above scheme, determining the calibration image for display on the screen includes: Determine the formula for generating the phase-shifted fringe pattern; The phase-shift fringe pattern is generated based on the generation formula of the phase-shift fringe pattern; The calibration image is generated based on the circular array and the phase-shifted fringe pattern.

[0008] In the above scheme, determining the phase value of each feature point based on the light intensity value, the amplitude, and the position coordinates of each feature point includes: Determine the pixel coordinates of the plane mirror; The coordinate index of each feature point is determined based on the pixel coordinates of the plane mirror and the position coordinates of each feature point. The phase value of each feature point is determined based on the coordinate index, the light intensity value, and the amplitude.

[0009] In the above scheme, determining the intrinsic parameter matrix of the camera, the distortion parameters of the camera, and the rotation and translation matrix between the camera and the display screen includes: The intrinsic parameter matrix and distortion parameters of the camera are obtained based on Zhang's calibration method. By introducing the Haushold transformation, a rigid body transformation formula between the display screen and the plane mirror is established; The rotation and translation matrix between the camera and the display screen is determined according to the rigid body transformation formula.

[0010] In the above scheme, the step of using a preset three-dimensional offset vector to correct the coordinates of each feature point to obtain the corrected coordinates of each feature point includes: Add constraints to the three-dimensional offset vector to obtain the three-dimensional offset vector with added constraints. Based on the three-dimensional offset vector after the additional constraints, the coordinates of each feature point are corrected to obtain the corrected coordinates of each feature point.

[0011] In the above scheme, the joint optimization of the deflection system based on the reprojection error to obtain the optimized intrinsic parameters of the camera and the optimized extrinsic parameters between the camera and the display screen includes: Using the reprojection error as the cost function and adding constraint formulas, a joint optimization formula is obtained; The deflection system is jointly optimized using the joint optimization formula to obtain the optimized intrinsic parameters of the camera and the optimized extrinsic parameters between the camera and the display screen.

[0012] According to one aspect of the embodiments of this application, a calibration device for a deflection system is proposed, which is applied to a deflection system including a display screen, a plane mirror, and a camera. The device includes: A first determining unit is configured to determine a calibration image for display on the screen, the calibration image consisting of a circular array and a phase-shifting stripe pattern, the circular array including a plurality of feature points; The second determining unit is used to determine the light intensity value and amplitude received by the camera for the calibration image, and to determine the phase value of each feature point based on the light intensity value, the amplitude and the position coordinates of each feature point. The third determining unit is used to determine the intrinsic parameter matrix of the camera, the distortion parameters of the camera, and the rotation and translation matrix between the camera and the display screen; The fourth determining unit is used to determine the initial intrinsic values ​​of the camera and the initial extrinsic values ​​between the camera and the display screen based on the intrinsic parameter matrix, the distortion parameters, and the rotation and translation matrix. The correction unit is used to correct the coordinates of each feature point using a preset three-dimensional offset vector to obtain the corrected coordinates of each feature point. The fifth determining unit is used to determine the reprojection error based on the position coordinates of each feature point and the corrected coordinates of each feature point, and to perform joint optimization of the deflection system based on the reprojection error to obtain the optimized intrinsic parameters of the camera and the optimized extrinsic parameters between the camera and the display screen.

[0013] The beneficial effects of this application are as follows: The deflection system of this application includes a display screen, a plane mirror, and a camera. First, a calibration image is determined on the display screen. The calibration image consists of a circular array and a phase-shifting fringe pattern, wherein the circular array includes multiple feature points, each of which has its specific position coordinates.

[0014] The light intensity and amplitude values ​​received by the camera when acquiring the calibration image are further determined. The phase value of each feature point can be calculated using the light intensity, amplitude, and position coordinates of each feature point. By matching the phase value with the feature points and the pixels of the display screen, a global position code is constructed without relying on the spatial distribution of a specific pattern. Even if the camera can only capture the center coordinates of some feature points, effective encoding of these feature points can still be achieved. Therefore, even under limited field of view conditions, the calibration process of the deflection system can proceed smoothly, ensuring high-precision geometric correction and feature matching.

[0015] By determining the intrinsic parameter matrix of the camera, the distortion parameters of the camera, and the rotation and translation matrix between the camera and the display screen, the deflection system can be initially calibrated, that is, the initial intrinsic parameters of the camera and the initial extrinsic parameters between the camera and the display screen can be determined. Due to the deformation error of the plane mirror and the display screen, this application uses a preset three-dimensional offset vector to correct the coordinates of each feature point, obtains the corrected coordinates of each feature point, and uses this to determine the reprojection error. By determining the obtained reprojection error, the deflection system is jointly optimized to obtain the optimized intrinsic parameters of the camera and the optimized extrinsic parameters between the camera and the display screen, thereby improving the calibration accuracy in the calibration process. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the deflection system provided in an embodiment of this application; Figure 2 A schematic flowchart illustrating the deflection system calibration method provided in this application embodiment; Figure 3 A schematic diagram of the calibration image provided in the embodiments of this application; Figure 4 A schematic diagram of the calibration process provided in the embodiments of this application; Figure 5 This is a comparison chart of reprojection error results without screen geometry compensation and reprojection error results with screen geometry compensation. Figure 6 The image shows a comparison of the reconstruction results of the plane mirror using system parameters obtained by the conventional calibration method and the calibration method proposed in this embodiment of the application, respectively. Figure 7A block diagram of the deflection system calibration device provided in the embodiments of this application. Detailed Implementation

[0017] To enable those skilled in the art to better understand the solutions of this application, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] It should be noted that while some processes described in the specification, claims, and accompanying drawings include multiple steps appearing in a specific order, it should be clearly understood that these steps may not be performed in the order they appear herein, or may be performed in parallel. The step numbers are merely used to distinguish different steps and do not themselves represent any execution order. Furthermore, descriptions such as "first," "second," or "objective" in this document are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. "Multiple" in this document refers to at least two.

[0019] It is worth noting that in the specific embodiments of this application, data such as calibration images and the position coordinates of feature points are involved. When the above embodiments of this application are applied to specific products or technologies, permission or consent from the target object is required, and the collection, use, and processing of related data must comply with relevant laws, regulations, and standards. For example, when an embodiment of this application needs to obtain data such as calibration images and the position coordinates of feature points, separate permission or consent from the target object can be obtained through pop-up windows or redirection to a confirmation page. After obtaining the separate permission or consent from the target object, the necessary calibration images, the position coordinates of feature points, and other related data required for the normal operation of the embodiment of this application can then be obtained.

[0020] The following provides a detailed description of the specific implementation methods of the embodiments of this application: See Figure 1 , Figure 1 This is a schematic diagram of the deflection system according to an embodiment of this application. Please refer to [link / reference]. Figure 2 , Figure 2 This is a flowchart illustrating the deflection system calibration method provided in this application embodiment. The deflection system calibration method of this application can be implemented by server 110 and / or terminal 140. Figure 2 The calibration method for the deflection system shown includes: Step 210: Determine a calibration image for display on the screen, the calibration image consisting of a circular array and a phase-shifting fringe pattern, the circular array including multiple feature points; Step 220: Determine the light intensity value and amplitude received by the camera for the calibration image, and determine the phase value of each feature point based on the light intensity value, the amplitude and the position coordinates of each feature point; Step 230: Determine the intrinsic parameter matrix of the camera, the distortion parameters of the camera, and the rotation and translation matrix between the camera and the display screen; Step 240: Determine the initial intrinsic values ​​of the camera and the initial extrinsic values ​​between the camera and the display screen based on the intrinsic parameter matrix, the distortion parameters, and the rotation and translation matrix; Step 250: Use a preset three-dimensional offset vector to correct the coordinates of each feature point to obtain the corrected coordinates of each feature point; Step 260: Determine the reprojection error based on the position coordinates of each feature point and the corrected coordinates of each feature point, and perform joint optimization on the deflection system based on the reprojection error to obtain the optimized intrinsic parameters of the camera and the optimized extrinsic parameters between the camera and the display screen.

[0021] The complete embodiment of this application will be explained in detail below with reference to steps 210-260: In step 210, a calibration image displayed on the screen is determined. This calibration image consists of a circular array and a phase-shifting fringe pattern, where the circular array includes multiple feature points. Since the geometric deformation of the screen is an inherent physical error, the feature points in the calibration pattern should reflect the fixed spatial distribution of the screen itself, and should not change with variations in the camera's viewing angle. Furthermore, due to limitations in the camera's field of view, a single captured reflection image typically only covers a portion of the screen. To ensure global matching and localization of feature points across multiple viewing angles, each feature point needs a globally unique position code. Considering these requirements, this embodiment adopts a combination of a circular array and a phase-shifting fringe pattern: the circular array assists the camera in extracting feature points and fixing the positions of screen feature points; the phase-shifting fringe pattern obtains the matching between image pixels and screen pixels through demodulation, and thus uses the phase value at the center of each circle to achieve position encoding of the center. It should be noted that the "circular array + phase-shifting stripe pattern" is only a preferred embodiment of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the inventive concept, such as "checkerboard + phase-shifting stripe pattern" and "circular markers + speckle pattern", which are all within the protection scope of the present invention.

[0022] like Figure 1 The image shown is an example of a deflection imaging system, where 2 is a display screen used for displaying... Figure 3The calibration pattern shown is received by camera 1 after being reflected by plane mirror 3. In this embodiment, the calibration pattern adopts a combination of "circular array + phase-shifting fringe pattern": the circular array is used to assist the camera in extracting feature points and fixing the position of screen feature points; the phase-shifting fringe pattern obtains the matching between image pixels and screen pixels through demodulation, and then the phase value at the center of the circle can be used to encode the position of each circle center.

[0023] Circular arrays, such as Figure 3 As shown in a, the phase-shifted fringe pattern is as follows: Figure 3 As shown in b, Figure 4 As shown, during the data acquisition phase, a standard plane mirror is placed on the opposite side of the camera and the screen (display screen) to reflect the calibration pattern described in step one. During the acquisition process, the plane mirror needs to change position multiple times, acquiring a set of images as described in step one at each position. After at least three acquisitions, the acquired screen mirror pattern is decoded to obtain the center coordinates of the mirror image at each position (i.e., the position coordinates of the feature points) and the corresponding screen physical coordinates (i.e., the pixel coordinates corresponding to the feature points).

[0024] Next, based on the center coordinates and corresponding screen physical coordinates, the camera intrinsic parameters are calibrated using Zhang's calibration method. The camera's intrinsic parameter matrix, distortion parameters, and the rotation and translation matrices between the screen mirror pattern acquired at each position and the camera are obtained. During this process, the screen is assumed to be an ideal plane.

[0025] In some embodiments, determining the calibration image for display on the screen includes: Determine the formula for generating the phase-shifted fringe pattern; The phase-shift fringe pattern is generated based on the generation formula of the phase-shift fringe pattern; The calibration image is generated based on the circular array and the phase-shifted fringe pattern.

[0026] Specifically, the formula for generating the phase-shifted fringe pattern in the embodiments of the present invention is as follows: (1) superscript A variable representing a member of the screen. This represents the average light intensity. Indicates amplitude. Represents screen pixel coordinates. Represents the phase value. Indicates the phase shift index. Control the phase shift of the cosine function.

[0027] In step 220, the light intensity value and amplitude received by the camera for the calibration image are determined, and the phase value of each feature point is determined based on the light intensity value, the amplitude and the position coordinates of each feature point.

[0028] Specifically, the pixel coordinates of the plane mirror are determined; The coordinate index of each feature point is determined based on the pixel coordinates of the plane mirror and the position coordinates of each feature point. The phase value of each feature point is determined based on the coordinate index, the light intensity value, and the amplitude.

[0029] The calibration image is reflected by the object under test and received by the camera. The light intensity value received by the camera can be expressed as: (2) superscript Represents the variable belonging to the camera. Represents the pixel coordinates of the imaging plane (i.e., the display screen); This represents the average light intensity received by the camera. This indicates the amplitude received by the camera; Indicated in pixels The phase value at that point. There are three unknowns in the equation that need to be solved, namely... , and Among them, the phase value The solution is generally obtained using the least squares method, that is: (3) Through the above process, the phase value of the pattern captured by the camera can be obtained. Due to the periodicity of the pattern, the phase value appears as a two-dimensional sawtooth wave in the image. To obtain a unique code between the camera and the screen, unwrapping is required to further remove periodic ambiguity.

[0030] After removing periodic ambiguities in the phase, each phase value calculated from the acquired image corresponds one-to-one with a pixel on the display screen. Then, by applying a center-of-circle extraction algorithm to obtain the center coordinates of the circular markers from the camera image, a match can be established between the phase values ​​and the display screen pixels, thus constructing a global position code. The advantage of this method is that it does not rely on the spatial distribution of a specific pattern; even if the camera can only capture the coordinates of some feature points, it can still effectively encode the positions of these center points. Therefore, even under limited field-of-view conditions, the system calibration process can still proceed smoothly, ensuring high-precision geometric correction and feature matching.

[0031] In step 230, the intrinsic parameter matrix of the camera, the distortion parameters of the camera, and the rotation and translation matrix between the camera and the display screen are determined.

[0032] Specifically, the camera's intrinsic parameter matrix and distortion parameters can be obtained based on Zhang's calibration method, while the rotation and translation matrices between the camera and the screen require the introduction of the Haushold transformation. This will be explained in detail below.

[0033] During the data acquisition phase, a standard plane mirror is placed on the opposite side of the camera and the screen to reflect the calibration pattern described in step one. During acquisition, the plane mirror needs to be repositioned multiple times, acquiring a set of images as described in step one at each position. After at least three acquisitions, the acquired screen mirror pattern is decoded as described in step one to obtain the center coordinates of the mirrored image at each position and the corresponding physical screen coordinates.

[0034] Next, based on the center coordinates and the corresponding screen physical coordinates, the camera intrinsic parameters are calibrated using Zhang's calibration method. This involves obtaining the camera's intrinsic parameter matrix, distortion parameters, and the rotation and translation matrices between the screen mirror pattern acquired at each location and the camera.

[0035] In step 240, the calibration of the rotation and translation matrix between the camera and the screen requires the introduction of the Haushold transformation to establish the rigid body transformation formula between the screen and its mirror image: (4) in and Let the rotation and translation matrices between the screen and the camera be... This represents the left-handed to right-handed coordinate system transformation matrix. and Indicates the first The rotation and translation matrix between the camera and the screen mirror at each position. and Indicates the first The normal of the plane mirror at each position and its distance from the camera, where This can be obtained through the perpendicular constraint between the plane intersection line and the normal line. The characteristic equation can be established and solved using the properties of the Haushold transformation.

[0036] Based on the above process, the initial values ​​of the extrinsic parameters between the camera and the screen can be obtained. It should be noted that the screen referred to in this embodiment is the same as the display screen.

[0037] In step 250, the coordinates of each feature point are corrected using a preset three-dimensional offset vector to obtain the corrected coordinates of each feature point. Specifically, constraints are added to the three-dimensional offset vector to obtain a constrained three-dimensional offset vector. Based on the constrained three-dimensional offset vector, the coordinates of each feature point are corrected to obtain the corrected coordinates of each feature point.

[0038] Here, due to errors in center coordinate extraction and data fitting during calibration, especially in multi-camera imaging systems, the initial calibration parameters obtained in step two inevitably contain deviations. These errors may be non-uniformly concentrated on certain parameters, leading to increased reprojection errors, unstable parameter estimation, and consequently affecting the overall calibration accuracy. Therefore, it is necessary to perform global optimization on the obtained initial values.

[0039] This application attempts to optimize screen geometric deformation during parameter optimization. As mentioned earlier, this application uses the center of the circle as a feature point. These feature points are discretely distributed on the screen. Under screen geometric deformation, these discrete feature points will deviate from the ideal planar position. This application uses a three-dimensional offset vector to represent the offset caused by geometric deformation. Therefore, the actual feature point coordinates should be: (5) in This represents the coordinates of the feature point in the screen coordinate system. This represents the three-dimensional offset vector resulting from geometric deformation. To avoid scale ambiguity, constraints are added to the three-dimensional offset vector: (6) In the formula This represents the total number of feature points.

[0040] In step 260, the reprojection error is determined by the position coordinates of each feature point and the corrected coordinates of each feature point, and the deflection system is jointly optimized based on the reprojection error to obtain the optimized intrinsic parameters of the camera and the optimized extrinsic parameters between the camera and the display screen.

[0041] Specifically, using reprojection error as the cost function and adding constraint formulas, the joint optimization formula for the deflection imaging system can be written as follows: (7) In the formula Indicates the first The position of the plane mirror at the first plane mirror position One feature point, Represents the camera intrinsic parameter matrix. and The radial distortion and tangential distortion parameters are separated. This represents the reprojection process.

[0042] Figure 5 To calibrate and compare the results, Figure 5 (a) in the figure represents the reprojection error result without compensation for screen geometry deformation; Figure 5 (b) in the figure represents the reprojection error result after incorporating screen geometry compensation. This allows for optimization of screen geometry during calibration, thereby improving the calibration accuracy of the intrinsic and extrinsic parameters of the deflection imaging system, all without the use of additional measuring equipment. Figure 5 It can be seen that the calibration error of the imaging system is reduced after screen geometric deformation compensation is added.

[0043] To verify the impact of calibration parameters on reconstruction quality, this invention uses a deflection imaging system to reconstruct a standard plane mirror and compares the reconstruction with that of a standard plane to obtain the reconstruction error. The results are as follows: Figure 6 As shown, Figure 6 (a) in the figure represents the error result of reconstruction using the traditional calibration method; Figure 6 Figure (b) shows the reconstruction error results using the calibration method proposed in this invention. As can be seen from the figure, the reconstruction results obtained using the calibration method proposed in this invention have better accuracy than those obtained using traditional methods.

[0044] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of the deflection system calibration device provided in the embodiments of this application. The deflection system calibration device is applied to computer equipment, and the deflection system calibration device may include: The first determining unit 701 is used to determine a calibration image for display on the screen, the calibration image being composed of a circular array and a phase-shifting stripe pattern, the circular array including a plurality of feature points; The second determining unit 702 is used to determine the light intensity value and amplitude received by the camera for the calibration image, and to determine the phase value of each feature point based on the light intensity value, the amplitude and the position coordinates of each feature point. The third determining unit 703 is used to determine the intrinsic parameter matrix of the camera, the distortion parameters of the camera, and the rotation and translation matrix between the camera and the display screen; The fourth determining unit 704 is used to determine the initial intrinsic values ​​of the camera and the initial extrinsic values ​​between the camera and the display screen based on the intrinsic parameter matrix, the distortion parameters, and the rotation and translation matrix. The correction unit 705 is used to correct the coordinates of each feature point using a preset three-dimensional offset vector to obtain the corrected coordinates of each feature point. The fifth determining unit 706 is used to determine the reprojection error based on the position coordinates of each of the feature points and the corrected coordinates of each of the feature points, and to perform joint optimization of the deflection system based on the reprojection error to obtain the optimized intrinsic parameters of the camera and the optimized extrinsic parameters between the camera and the display screen.

[0045] Furthermore, the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or apparatus.

[0046] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0047] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0048] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0049] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0050] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0051] The above is a detailed description of the embodiments of this application. However, this application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A deflection system calibration method, characterized by, The method is applied to a deflection system, the deflection system comprising a display screen, a plane mirror and a camera, and the method comprising: determining a calibration image for display on the display screen, the calibration image comprising a circular array and a phase-shift fringe pattern, the circular array comprising a plurality of feature points; determining light intensity values and amplitudes received by the camera for the calibration image, and determining phase values of the feature points according to the light intensity values, the amplitudes and position coordinates of the feature points; determining an intrinsic matrix of the camera, distortion parameters of the camera and a rotation and translation matrix between the camera and the display screen; determining initial values of the intrinsic matrix of the camera and initial values of extrinsic parameters between the camera and the display screen based on the intrinsic matrix, the distortion parameters and the rotation and translation matrix; performing coordinate correction on the feature points using a preset three-dimensional offset vector to obtain corrected coordinates of the feature points; determining re-projection errors based on the position coordinates of the feature points and the corrected coordinates of the feature points, and performing joint optimization on the deflection system based on the re-projection errors to obtain optimized intrinsic parameters of the camera and optimized extrinsic parameters between the camera and the display screen.

2. The method of claim 1, wherein The method comprises: determining a generation formula of the phase-shift fringe pattern; generating the phase-shift fringe pattern based on the generation formula of the phase-shift fringe pattern; generating the calibration image based on the circular array and the phase-shift fringe pattern.

3. The method of claim 1, wherein The method comprises: determining pixel coordinates of the plane mirror; determining coordinate indexes of the feature points based on the pixel coordinates of the plane mirror and the position coordinates of the feature points; determining the phase values of the feature points based on the coordinate indexes, the light intensity values and the amplitudes.

4. The method of claim 1, wherein The method comprises: obtaining the intrinsic matrix of the camera and the distortion parameters of the camera based on Zhang's calibration method; establishing a rigid transformation formula between the display screen and the plane mirror by introducing a Householder transformation; determining the rotation and translation matrix between the camera and the display screen according to the rigid transformation formula.

5. The method of claim 1, wherein The method comprises: appending constraints to the three-dimensional offset vector to obtain an appended three-dimensional offset vector; performing coordinate correction on the feature points based on the appended three-dimensional offset vector to obtain the corrected coordinates of the feature points.

6. The method of claim 1, wherein The method comprises: taking the re-projection errors as a cost function and appending a constraint formula to obtain a joint optimization formula. The deflection system is jointly optimized by the joint optimization formula, to obtain optimized intrinsic parameters of the camera and optimized extrinsic parameters between the camera and the display screen.

7. A deflection system calibration apparatus, characterized by, The device is applied to a deflection system, and the deflection system comprises a display screen, a plane mirror and a camera. A first determination unit is configured to determine a calibration image displayed on the display screen, the calibration image is composed of a circular array and a phase shift fringe pattern, and the circular array comprises a plurality of feature points. A second determination unit is configured to determine light intensity values and amplitude values received by the camera for the calibration image, and determine phase values of the feature points according to the light intensity values, the amplitude values and position coordinates of the feature points. A third determination unit is configured to determine an intrinsic parameter matrix of the camera, distortion parameters of the camera and rotation and translation matrices between the camera and the display screen. A fourth determination unit is configured to determine initial values of intrinsic parameters of the camera and initial values of extrinsic parameters between the camera and the display screen based on the intrinsic parameter matrix, the distortion parameters and the rotation and translation matrices. A correction unit is configured to correct coordinates of the feature points by using a preset three-dimensional offset vector, to obtain corrected coordinates of the feature points. A fifth determination unit is configured to determine re-projection errors based on the position coordinates of the feature points and the corrected coordinates of the feature points, and jointly optimize the deflection system based on the re-projection errors, to obtain optimized intrinsic parameters of the camera and optimized extrinsic parameters between the camera and the display screen.

Images