OST calibration system and OST calibration method

Abstract

The invention relates to an OST calibration system and an OST calibration method.The OST calibration system is used for calibrating OST-AR equipment, the OST-AR equipment comprises an AR display screen and a tracking camera which are correspondingly configured, and the OST calibration system comprises a three-dimensional target plate on which multiple sets of feature points are distributed; the industrial camera is used for simulating human eyes; the baffle is provided with a first position for shielding the three-dimensional target plate for the industrial camera and a second position for relatively avoiding the three-dimensional target plate; the industrial personal computer is used for acquiring virtual body target plate data and entity target plate data and correspondingly calculating to obtain calibration parameters of the OST-AR equipment, the virtual body target plate data is obtained by shooting an image target plate displayed by the AR display screen by the industrial camera when the baffle plate is at the first position, and the entity target plate data is obtained by shooting the image target plate displayed by the AR display screen when the baffle plate is at the second position. The entity target plate data is obtained by shooting the stereoscopic target plate through the AR display screen by the industrial camera and shooting the stereoscopic target plate through the tracking camera when the baffle plate is at the second position.

Description

Technical Field

[0001] The present application relates to the technical field of OST calibration, and in particular to an OST calibration system and an OST calibration method. Background Art

[0002] Augmented reality (AR) is an interactive real-time technology that allows users to feel that virtual objects exist between real objects in the physical world. Currently, there are two main display modes for AR helmet displays, video see-through (VST) and optical see-through (OST). Although VST is easy to deploy, the human eye obtains video stream information, and the camera's observation viewpoint does not coincide with the actual viewpoint of the human eye. In comparison, the visual experience brought by the OST display mode is more natural.

[0003] With the advancement of technology, the hardware of OST helmet displays has become smaller and lighter. With the support of various sensors and high-performance processors, virtual images can be calculated and rendered in real time. At present, OST helmet displays have been widely used in entertainment, education, medical and other fields.

[0004] However, in OST helmet displays, because the virtual content must be changed according to the position of the human eye, it is difficult to correctly align the virtual content with the real world, so OST calibration is required.

[0005] At present, OST calibration is mainly a single-point active alignment method, the principle of which is: fix several markers in the real scene, the user observes the markers through the projection screen, and uses the mouse to control the screen aimer to align the markers. However, once the device slides or the user changes, recalibration is required. Therefore, the common calibration method is to divide the single-point active alignment method into two parts: offline calibration and user online calibration, which is called the two-step method.

[0006] The existing two-step method needs to adjust the physical target plate during offline calibration. The calibration process is cumbersome and time-consuming. In addition, affected by the structure of the physical target plate and the calibration algorithm, the calibration reprojection error needs to be optimized. Summary of the invention

[0007] The present application provides an OST calibration system and an OST calibration method which can improve the degree of automation and versatility.

[0008] The present application provides an OST calibration system for calibrating an OST-AR device, wherein the OST-AR device includes an AR display screen and a tracking camera configured accordingly. The OST calibration system includes:

[0009] Three-dimensional target plate, with multiple groups of feature points distributed;

[0010] Industrial cameras, used to simulate the human eye;

[0011] The baffle has a first position for shielding the three-dimensional target plate from the industrial camera and a second position for relatively avoiding the target plate;

[0012] The industrial computer is used to obtain virtual target plate data and physical target plate data and calculate the calibration parameters of the OST-AR device accordingly, wherein the virtual target plate data is obtained by photographing the image target plate displayed on the AR display screen by the industrial camera when the baffle is in the first position, and the physical target plate data is obtained by photographing the stereoscopic target plate through the AR display screen by the industrial camera when the baffle is in the second position, and by photographing the stereoscopic target plate by the tracking camera.

[0013] Several optional methods are also provided below, but they are not intended to be additional limitations on the above-mentioned overall solution, but are merely further supplements or preferences. Under the premise that there are no technical or logical contradictions, each optional method can be combined with the above-mentioned overall solution separately, and multiple optional methods can also be combined.

[0014] In one embodiment, the three-dimensional target plate includes a plurality of plane target plates facing different directions and intersecting with each other, each group of feature points is distributed on a corresponding plane target plate and each feature point is circular.

[0015] In one embodiment, the OST calibration system further includes a carrier device for mounting the OST-AR device, and the industrial camera is mounted on the carrier device;

[0016] The carrying device is provided with an adjusting mechanism controlled by the industrial computer and used for changing the three-dimensional spatial posture of the carrying device.

[0017] In one embodiment, there are two industrial cameras and they move independently of each other.

[0018] In one embodiment, the three-dimensional target plate and the industrial camera are respectively provided with six-axis adjustment devices controlled by the industrial computer for changing their three-dimensional spatial postures.

[0019] In one embodiment, the baffle is provided with a switching mechanism controlled by the industrial computer, and is used for driving the baffle to switch between the first position and the second position.

[0020] The present application also provides an OST calibration method, including:

[0021] Step S100, respectively providing an OST-AR device to be calibrated, an industrial camera, a three-dimensional target plate and a baffle, wherein the three-dimensional target plate has feature points, the OST-AR device includes an AR display screen and a tracking camera configured accordingly, and the baffle has a first position for shielding the three-dimensional target plate from the industrial camera and a second position for relatively avoiding the three-dimensional target plate;

[0022] Step S200, when the baffle is at the first position, photographing the image target displayed on the AR display screen via the industrial camera to obtain virtual target data;

[0023] Step S300, when the baffle is in the second position, photographing the three-dimensional target plate through the industrial camera through the AR display screen and through the tracking camera to obtain physical target plate data;

[0024] Step S400: Calculate and obtain calibration parameters of the OST-AR device according to the virtual target plate data and the physical target plate data.

[0025] In one embodiment, in step S400, calculating and obtaining the calibration parameters of the OST-AR device includes:

[0026] Step S410: extracting feature points from the virtual body target plate data to obtain first information, where the first information is two-dimensional coordinate information of the virtual body feature points in the industrial camera imaging coordinate system;

[0027] Step S420: obtaining a homography relationship between imaging planes of the AR display screen and the industrial camera according to the first information and feature point information in the image target plate;

[0028] Step S430, extracting feature points from the entity target data to obtain second information and third information, wherein the second information is two-dimensional coordinate information of the entity feature points in the industrial camera imaging coordinate system, and the third information is two-dimensional coordinate information of the entity feature points in the tracking camera imaging coordinate system;

[0029] Step S440: Obtain fourth information according to the second information and the homography relationship, where the fourth information is two-dimensional coordinate information of the entity feature point in the AR display screen imaging coordinate system;

[0030] Step S450, obtaining three-dimensional coordinate information of the entity feature point in the imaging coordinate system of the tracking camera according to the third information, the physical size of the entity feature point, and the intrinsic parameters and distortion parameters of the tracking camera;

[0031] Step S460: Obtain the calibration parameters according to the fourth information and the three-dimensional coordinate information.

[0032] In one embodiment, the feature point is a circle, and the step S410 further includes reading the intrinsic parameters and distortion parameters of the industrial camera, and performing a dedistortion operation on the virtual feature point according to the intrinsic parameters and distortion parameters of the industrial camera to obtain the first information.

[0033] In one embodiment, in step S460, the calibration parameters include external parameters and a projection matrix of the OST-AR device, wherein the projection matrix is ​​an asymmetric projection matrix.

[0034] The OST calibration system and OST calibration method of the present application can be used to calibrate AR head-mounted display devices. A clearer image can be obtained through the baffle. In addition, the use of the baffle makes it easier to adjust the position, adapt to different calibration scenarios, and improve the versatility of the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 This is a structural block diagram of an OST calibration system in one embodiment of the present application;

[0037] Figure 2 This is a schematic diagram of the structure of some hardware of the OST calibration system in one embodiment of the present application;

[0038] Figure 3 for Figure 2 Schematic diagram of the structure of the central stereoscopic mark plate;

[0039] Figure 4 This is a functional module diagram of the OST calibration software in one embodiment of the present application;

[0040] Figure 5 This is a flow chart of an OST calibration method in one embodiment of the present application;

[0041] Figure 6 for Figure 4 Specific flow chart of step S400;

[0042] Figure 7 This is a schematic diagram of the calculation of the moving position of the three-dimensional target;

[0043] Figure 8 A schematic diagram of the interface of the reprojection error after implementing the OST calibration method of this application;

[0044] Fig. 9 for Figure 8 A magnified view of one of the feature points in .

[0045] The reference numerals of the components are as follows:

[0046] 1. Display; 2. Industrial computer; 3. Six-axis motor; 4. Six-axis motor; 5. Telescopic rod; 6. Six-axis motor; 7. Industrial camera; 8. Industrial camera; 9. AR head display device; 10. Tracking camera; 11. Baffle; 12. Three-dimensional label. DETAILED DESCRIPTION

[0047] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0048] It should be noted that when a component is referred to as being "fixed to" or "disposed on" another component, it may be directly on the other component or there may also be a central component. When a component is considered to be "connected to" another component, it may be directly connected to the other component or there may be a central component at the same time. The terms "vertical", "horizontal", "upper", "lower", "left", "right" and similar expressions used in the specification of this application are for illustrative purposes only and do not represent the only implementation method.

[0049] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0050] In the present application, unless otherwise clearly specified and limited, a first feature being “above” or “below” a second feature may mean that the first feature is directly in contact with the second feature, or the first feature and the second feature are indirectly in contact through an intermediate medium. Moreover, a first feature being “above”, “above”, or “above” a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level (or in a state of use, or from a certain perspective of the drawing) than the second feature. A first feature being “below”, “below”, or “below” a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level (or in a state of use, or from a certain perspective of the drawing) than the second feature.

[0051] Unless otherwise defined, all technical and scientific terms used in the specification of this application have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in the specification of this application includes any and all combinations of one or more related listed items.

[0052] In the existing technology, during the OST offline calibration process, the target plate needs to be located in the field of view of both the AR head display device tracking camera and the industrial camera at the same time to extract feature points. However, currently these operations require manual control, which is not only cumbersome and relies on the operator's experience, but also cannot guarantee that each adjustment can meet the calibration requirements. At present, the automated calibration system and equipment related to OST offline calibration are still in the development and improvement stage, and there is a lack of system-level automated calibration equipment.

[0053] The following is a description of the relevant terms:

[0054] VR: Virtual Reality.

[0055] AR: Augmented Reality.

[0056] OST: Optical See Through, optical perspective, through a semi-transparent and semi-reflective optical system, while seeing the real world, the virtual image is reflected and superimposed into the field of vision.

[0057] VST: Video See Through, video perspective, the camera captures the video stream of the scene, superimposes virtual information into the video stream, and finally renders the processed video stream frame by frame on the display.

[0058] OST calibration: Calibrate the tracking camera and AR display of the OST-AR device to ensure the alignment and accurate projection of the virtual image with the real world.

[0059] See also Figure 1 to Figure 3 An embodiment of the present application provides an OST calibration system for calibrating an OST-AR device (i.e., an AR head display device 9), wherein the OST-AR device includes a correspondingly configured AR display screen and a tracking camera 10, and the tracking camera 10 is used to capture a stereoscopic target plate for data collection during OST calibration. The OST-AR device supports a screen projection function, and the image target plate captured by the tracking camera 10 can be projected onto the AR display screen.

[0060] The OST calibration system of this embodiment includes a three-dimensional target plate 12, an industrial camera and a baffle 11. In order to install the OST-AR device during calibration, a carrying device may be configured. The carrying device may adopt a head mold or other supporting structure. In order to change the three-dimensional spatial posture of the OST-AR device, the carrying device may be configured with an adjustment mechanism controlled by the industrial computer 2.

[0061] Industrial cameras are used to simulate human eyes. In this embodiment, there are two industrial cameras that simulate the left eye and the right eye respectively. For example, industrial camera 7 is used to collect virtual and physical target plate data at the left eye position in OST calibration, and industrial camera 8 is used to collect virtual and physical target plate data at the right eye position in OST calibration.

[0062] The two industrial cameras are respectively equipped with six-axis motors, for example, industrial camera 7 is equipped with six-axis motor 3, and industrial camera 8 is equipped with six-axis motor 4. Each six-axis motor is composed of a three-axis mobile motor and a three-axis rotary motor, which is used to control the corresponding industrial camera to move with six degrees of freedom. The two industrial cameras can be installed on the carrier device, that is, the OST-AR device moves with the carrier device as a whole, but the two industrial cameras can move independently relative to the carrier device.

[0063] The three-dimensional target plate 12 is roughly a cubic structure, and is equipped with a six-axis motor 6 controlled by the industrial computer 2, which is used to change the three-dimensional spatial posture of the three-dimensional target plate 12. The same type of plane target plates are arranged on three adjacent faces (two by two are perpendicular) of the three-dimensional target plate 12, so that the industrial camera and the tracking camera 10 can simultaneously collect image information of multiple plane target plates.

[0064] Since the three-dimensional target plate 12 has three adjacent plane target plates, one of the target plates can be set as the three-dimensional target plate setting surface for convenience of calculation, such as Figure 3 As shown in the dotted box. Each plane target plate is distributed with a set of feature points. As one of the improvements, the feature points used in this application are all circular and arranged in rows and columns. The feature points between adjacent rows are staggered with each other. The use of circular feature points is more conducive to recognition, calculation and optimization of the final calibration parameters, which can further reduce the reprojection error.

[0065] The baffle 11 is used to block objects behind the AR display screen of the AR head display device 9. The baffle 11 can be configured with a switching mechanism controlled by the industrial computer 2. For example, the telescopic rod 5 or other driving methods can be used to control the up and down movement of the baffle, so that the baffle 11 can switch between the first position and the second position. When the baffle 11 is in the first position, the baffle 11 blocks objects behind the AR display screen and at least blocks the three-dimensional target plate 12, preventing the optical image of the three-dimensional target plate 12 from directly passing through the AR display screen into the industrial camera, thereby facilitating the industrial camera to shoot the image target plate displayed on the AR display screen and collect clear virtual target plate data at the left and right eye positions.

[0066] The baffle 11 avoids the three-dimensional target plate 12 in the second position, so that the optical image of the three-dimensional target plate 12 directly enters the industrial camera through the AR display screen, which is convenient for the industrial camera to shoot the three-dimensional target plate 12 through the AR display screen and obtain clear physical target plate data.

[0067] The baffle 11 in this embodiment is not only simple in structure and flexible in operation, but also has no strict restrictions on its own shape and the movement mode of switching positions. A six-axis motor can be provided when necessary to adapt to the posture changes of the AR head display device 9, thereby ensuring the shielding effect and facilitating the clarity of image acquisition. The baffle 11 and the AR head display device 9 can both adapt to each other and be separated from each other for easy adjustment, further improving the versatility of the device.

[0068] The industrial computer 2 is used as a carrier for the control, data collection, storage, command sending, data processing and software algorithm of multiple cameras and various moving parts. The industrial computer 2 can be connected to a display 1, through which the preview of each camera, software interface and data visualization are performed.

[0069] During the calibration process, the industrial computer 2 obtains the virtual target plate data and the physical target plate data and calculates and obtains the calibration parameters of the OST-AR device accordingly.

[0070] Figure 2 Taking the left eye calibration as an example, during OST calibration, the tracking camera 10 and the industrial camera 7 need to simultaneously capture the stereoscopic target plate image. According to the correspondence between the feature points, the OST external parameters from the tracking camera 10 to the left eye point (i.e., the external parameters of the OST-AR device) and the projection matrix of the left eye point on the AR projection screen are calibrated.

[0071] The virtual target plate data is obtained by photographing the image target plate displayed on the AR display screen by an industrial camera when the baffle 11 is in the first position, and the physical target plate data is obtained by photographing the three-dimensional target plate by an industrial camera through the AR display screen and by photographing the three-dimensional target plate by the tracking camera 10 when the baffle 11 is in the second position.

[0072] The software module is also configured in the industrial computer 2 accordingly, namely Figure 4 The OST calibration software in the system can include the following functional modules:

[0073] Logic control (module): used to send control commands to each camera, control the extension and retraction of the telescopic rod, and control the AR head display device to turn on and off the projection mode.

[0074] Motion control (module): The position and posture of each six-axis motor are calculated and the movement of each six-axis motor is controlled accordingly.

[0075] Algorithm call (module): input the collected data into the OST calibration algorithm for calculation.

[0076] Result visualization (module): The results of algorithm calibration are visualized on the software side for users to view.

[0077] The OST calibration system of the present application can set a plane target mode and a stereo target mode to adapt to the tracking cameras of different AR head-mounted display devices, and has good scalability. For example, the stereo target mode is suitable for situations where the distortion of the AR head-mounted display device tracking camera is small, and the plane target mode is suitable for situations where the distortion of the AR head-mounted display device tracking camera is large.

[0078] The automated control of the 3D target movement reduces human operations to a great extent and improves calibration efficiency. For example, the six-axis end-point pose increment when the 3D target moves to the target pose can be directly calculated through pose conversion to achieve automated adjustment of the calibration.

[0079] The industrial cameras used in the OST calibration system of this application support lenses with different fovs, and each camera has an independent motion mechanism that can be used for posture adjustment. It can be adapted to calibrate different optical machine fovs and different forms of equipment, and the equipment adaptability is high. The two industrial cameras have independent posture control modules in control, which can be adapted to calibrate equipment of different forms.

[0080] See also Figure 5-6 An embodiment of the present application further provides an OST calibration method, which can be implemented by the OST calibration system of any of the above embodiments. The OST calibration method of this embodiment specifically includes:

[0081] Step S100, respectively provide an OST-AR device to be calibrated, an industrial camera, a three-dimensional target plate and a baffle, wherein the three-dimensional target plate has feature points, the OST-AR device includes a correspondingly configured AR display screen and a tracking camera, and the baffle has a first position for shielding the three-dimensional target plate from the industrial camera and a second position for relative avoidance.

[0082] Step S200: When the baffle is at the first position, the image target displayed on the AR display screen is photographed by an industrial camera to obtain virtual target data.

[0083] Step S300: When the baffle is at the second position, the three-dimensional target plate is photographed by the industrial camera through the AR display screen and by the tracking camera to obtain physical target plate data.

[0084] Step S400: Calculate and obtain calibration parameters of the OST-AR device based on the virtual target plate data and the physical target plate data.

[0085] In one embodiment, in step S400, calculating and obtaining calibration parameters of the OST-AR device includes:

[0086] Step S410: extract feature points from the virtual body target plate data to obtain first information, where the first information is two-dimensional coordinate information of the virtual body feature points in the industrial camera imaging coordinate system.

[0087] The virtual feature points are obtained by extracting feature points from the virtual target plate data. In this step, in order to further optimize the calibration effect, the intrinsic parameters and distortion parameters of the industrial camera are read, and the first information is obtained after dedistortion operation is performed on the virtual feature points according to the intrinsic parameters and distortion parameters of the industrial camera.

[0088] Step S420: Obtain a homography relationship between the imaging planes of the AR display screen and the industrial camera according to the first information and the feature point information in the image target plate. For details, see the following formula:

[0089]

[0090] in:

[0091] is the feature point information in the image target, that is, the pixel coordinates of the virtual feature point in the imaging plane of the AR display screen;

[0092] P 2D (testCam) is the first information;

[0093] H(testCam2display) is the homography relationship from the imaging plane of the industrial camera to the imaging plane of the AR display screen.

[0094] In order to adapt to different AR devices, the selected industrial camera supports different FOV lenses. Since the image captured by the industrial camera is dedistorted in step S410, the error of OST calibration is reduced.

[0095] Step S430, extract feature points from the entity target data to obtain second information and third information, where the second information is the two-dimensional coordinate information of the entity feature points in the industrial camera imaging coordinate system, and the third information is the two-dimensional coordinate information of the entity feature points in the tracking camera imaging coordinate system.

[0096] Step S440: Obtain fourth information according to the second information and the homography relationship, where the fourth information is two-dimensional coordinate information of the entity feature point in the AR display screen imaging coordinate system.

[0097] Step S450: Obtain the three-dimensional coordinate information of the entity feature point in the imaging coordinate system of the tracking camera according to the third information, the physical size of the entity feature point, and the intrinsic parameters and distortion parameters of the tracking camera.

[0098] Step S460: obtain calibration parameters according to the fourth information and the three-dimensional coordinate information. The calibration parameters include the external parameters and projection matrix of the OST-AR device. Taking the left eye (right eye) optimization as an example, the following relationship can be seen:

[0099]

[0100] in:

[0101] residual is the residual, that is, the reprojection error (ideally 0);

[0102] For the fourth information;

[0103] P 3D (RGB) is the three-dimensional coordinate information of the entity feature point in the tracking camera imaging coordinate system;

[0104] M is the projection matrix of the OST-AR device;

[0105] T(RGB2Lefteye) is the external parameter of the OST-AR device, that is, the pose transformation matrix from the tracking camera coordinate system to the left eye point.

[0106] Since the left and right industrial cameras are controlled separately in this application, their optical axes are not strictly consistent with the center normal of the AR projection screen. Therefore, the improved algorithm optimizes the projection matrix M in the form of an asymmetric projection matrix, reducing the error of OST calibration.

[0107] It should be understood that, although the steps in the relevant flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0108] In one implementation, the specific steps for performing OST calibration are as follows:

[0109] All powered components are powered on normally, and the six-axis motor returns to the preset initial position so that the set surface of the three-dimensional target plate faces the industrial camera.

[0110] Open the OST calibration software on the industrial computer and enter the model of the AR headset to be tested.

[0111] Turn on the AR display screen of the AR headset to be tested, place the AR headset on the head model, and adjust the headband to ensure that it is firmly worn and not loose.

[0112] In the software interface, input the internal parameters of the two industrial cameras and the tracking camera, the type and feature point parameters of the virtual target (image target) and the three-dimensional target, the image resolution of the virtual target, the pixel coordinates of the initial feature points and other parameters.

[0113] Click the Auto-calibration button and the OST calibration software will start running.

[0114] The OST calibration software controls the telescopic rod to extend, causing the baffle to rise, so that the industrial camera can clearly capture the AR display screen.

[0115] The OST calibration software controls the six-axis motors to adjust the two industrial cameras so that they can capture the virtual target plate on the AR display screen completely and clearly.

[0116] The OST calibration software controls the industrial camera to collect and save virtual target images (virtual target data) at the positions of the left and right eyes.

[0117] The OST calibration software controls the AR headset to turn off screen projection.

[0118] The OST calibration software controls the telescopic rod to retract, causing the baffle to drop.

[0119] The OST calibration software controls the industrial camera to collect and save the image of the current stereo target surface.

[0120] The OST calibration software runs the six-axis motor end mobile posture calculation algorithm, and calculates the posture change of the six-axis motor end through the hand-eye calibration external parameters of the stereo target setting surface and the six-axis motor end, the external parameters of the current stereo target setting surface relative to the industrial camera, and the external parameters of the stereo target setting surface relative to the industrial camera under the ideal OST calibration posture.

[0121] like Figure 7 As shown, where:

[0122] P is the origin of the base coordinate system of the six-axis adjustment device (six-axis motor) of the three-dimensional target plate;

[0123] a0 is the current position of the end of the six-axis adjustment device of the three-dimensional target plate, and correspondingly, b0 is the corresponding position of the origin of the coordinate system of the setting surface of the three-dimensional target plate;

[0124] a1 is the end position of the six-axis adjustment device under the target posture of the three-dimensional target plate, and correspondingly, b1 is the corresponding position of the origin of the coordinate system of the setting surface of the three-dimensional target plate;

[0125] The dotted line on the left shows the initial position of the industrial camera, and below it is the position where the industrial camera collects virtual target data, where c is the origin of the industrial camera coordinate system.

[0126] According to the diagram, the corresponding relationships mentioned above can be obtained, for example, the corresponding relationship between c and b0 is Tc_b0; the corresponding relationship between c and b1 is Tc_b1; the corresponding relationship between a0 and b0 is Ta_b, the corresponding relationship between a1 and b1 is Ta_b; the corresponding relationship between a0 and a1 is Ta0_a1.

[0127] The OST calibration software runs the six-axis mechanism motion control algorithm and controls the movement of the six-axis motor according to the change in the end position and posture, so that the three-dimensional target plate moves to the ideal OST calibration posture.

[0128] The OST calibration software controls the industrial camera and the tracking camera to acquire and save the current stereo target image (stereo target data).

[0129] The OST calibration software runs the OST calibration algorithm, uses industrial cameras and tracking cameras to collect the correspondence between feature points in the stereo target plate, calculates the OST external parameters and projection matrix of the AR headset device, saves the OST calibration results locally, and visualizes the OST calibration results on the display.

[0130] Close the OST calibration software or click the reset button to reset the motor to the preset initial position.

[0131] Remove the AR headset from the head mold, turn off the industrial computer, and power off the device.

[0132] At this point, the OST automated calibration system has completed all calibration items, including OST extrinsic parameters and projection matrix.

[0133] like Figure 8 , Fig. 9 As shown, Figure 8 The number in the upper left corner is the average reprojection error, and the number above each feature point is the feature point reprojection error. The error unit is points. In this accuracy verification result, the maximum reprojection error of this application does not exceed 2 points.

[0134] The OST calibration system of this application has a high degree of automation, which improves the OST calibration efficiency and lowers the operating threshold. The equipment is almost not affected by human factors, can achieve calibration results with good consistency, and is convenient for standardized implementation. In addition, it has good scalability and can perform OST calibration for most products with high calibration accuracy.

[0135] The technical features of the above-mentioned embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. When the technical features in different embodiments are embodied in the same figure, it can be regarded that the figure also discloses the combination examples of the various embodiments involved.

[0136] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of patent protection of the present application shall be subject to the attached claims.

Claims

1. An OST calibration system for calibrating an OST-AR device, wherein the OST-AR device comprises an AR display screen and a tracking camera configured accordingly, characterized in that: The OST calibration system includes: Three-dimensional target plate, with multiple groups of feature points distributed; Industrial cameras, used to simulate the human eye; The baffle has a first position for shielding the three-dimensional target plate from the industrial camera and a second position for relatively avoiding the target plate; The industrial computer is used to obtain virtual target plate data and physical target plate data and calculate the calibration parameters of the OST-AR device accordingly, wherein the virtual target plate data is obtained by photographing the image target plate displayed on the AR display screen by the industrial camera when the baffle is in the first position, and the physical target plate data is obtained by photographing the stereoscopic target plate through the AR display screen by the industrial camera when the baffle is in the second position, and by photographing the stereoscopic target plate by the tracking camera.

2. The OST calibration system according to claim 1, characterized in that: The three-dimensional target plate includes a plurality of plane target plates facing different directions and intersecting each other, each group of feature points is distributed on a corresponding plane target plate and each feature point is circular.

3. The OST calibration system according to claim 1, characterized in that: The OST calibration system further comprises a carrying device for installing the OST-AR device, and the industrial camera is installed on the carrying device; The carrying device is provided with an adjusting mechanism controlled by the industrial computer and used for changing the three-dimensional spatial posture of the carrying device.

4. The OST calibration system according to claim 3, characterized in that: There are two industrial cameras and they move independently of each other.

5. The OST calibration system according to claim 1, characterized in that: The three-dimensional target plate and the industrial camera are respectively provided with six-axis adjustment devices controlled by the industrial computer for changing their three-dimensional spatial postures.

6. The OST calibration system according to claim 1, characterized in that: The baffle is configured with a switching mechanism controlled by the industrial computer, which is used to drive the baffle to switch between the first position and the second position.

7. An OST calibration method, characterized in that: include: Step S100, respectively providing an OST-AR device to be calibrated, an industrial camera, a three-dimensional target plate and a baffle, wherein the three-dimensional target plate has feature points, the OST-AR device includes an AR display screen and a tracking camera configured accordingly, and the baffle has a first position for shielding the three-dimensional target plate from the industrial camera and a second position for relatively avoiding the three-dimensional target plate; Step S200, when the baffle is at the first position, photographing the image target displayed on the AR display screen via the industrial camera to obtain virtual target data; Step S300, when the baffle is in the second position, photographing the three-dimensional target plate through the industrial camera through the AR display screen and through the tracking camera to obtain physical target plate data; Step S400: Calculate and obtain calibration parameters of the OST-AR device according to the virtual target plate data and the physical target plate data.

8. The OST calibration method according to claim 7, characterized in that: In step S400, calculating and obtaining the calibration parameters of the OST-AR device includes: Step S410: extracting feature points from the virtual body target plate data to obtain first information, where the first information is two-dimensional coordinate information of the virtual body feature points in the industrial camera imaging coordinate system; Step S420: obtaining a homography relationship between imaging planes of the AR display screen and the industrial camera according to the first information and feature point information in the image target plate; Step S430, extracting feature points from the entity target data to obtain second information and third information, wherein the second information is two-dimensional coordinate information of the entity feature points in the industrial camera imaging coordinate system, and the third information is two-dimensional coordinate information of the entity feature points in the tracking camera imaging coordinate system; Step S440: Obtain fourth information according to the second information and the homography relationship, where the fourth information is two-dimensional coordinate information of the entity feature point in the AR display screen imaging coordinate system; Step S450, obtaining three-dimensional coordinate information of the entity feature point in the imaging coordinate system of the tracking camera according to the third information, the physical size of the entity feature point, and the internal parameters and distortion parameters of the tracking camera; Step S460: Obtain the calibration parameters according to the fourth information and the three-dimensional coordinate information.

9. The OST calibration method according to claim 8, characterized in that: The feature point is a circle. The step S410 further includes reading the internal parameters and distortion parameters of the industrial camera, and performing a dedistortion operation on the virtual feature point according to the internal parameters and distortion parameters of the industrial camera to obtain the first information.

10. The OST calibration method according to claim 8, characterized in that: In step S460, the calibration parameters include the external parameters and projection matrix of the OST-AR device, wherein the projection matrix is ​​an asymmetric projection matrix.

Images