Eye-tracking module calibration equipment

Abstract

This application provides an eye-tracking module calibration device comprising: a cabinet including a device frame, an electrical base plate fixed to the device frame, a workbench fixed to the device frame and located above the electrical base plate, and an industrial control computer mounted on the electrical base plate; a device fixture mounted on the workbench for fixing the device to be calibrated; a functional calibration plate assembly including a planar calibration plate with a characteristic pattern and a spherical calibration plate with one or more reflective spheres; and a robotic arm assembly mounted in the cabinet and communicatively connected to the industrial control computer, capable of selectively moving the planar calibration plate or the spherical calibration plate in the functional calibration plate assembly to multiple calibration points.

Description

Technical Field

[0001] This application relates to the field of eye-tracking technology, and in particular to an eye-tracking module calibration device. Background Technology

[0002] In recent years, with the widespread application of eye-tracking technology in various fields such as virtual reality, augmented reality, psychological research and medical diagnosis, eye tracking based on pupil-corneal reflection (PCR) has become a common and efficient method. By analyzing the characteristics of reflected light on the surface of the eyeball and combining it with the capture of pupil and corneal reflection points by an ET camera, it can accurately measure the movement trajectory of the eyeball.

[0003] In this eye-tracking system, the intrinsic ET parameter provides the conversion between 2D features of the ET camera image and 3D features in the physical coordinate system, while the extrinsic ET parameter provides the position of the ET light source, such as an LED, in the physical coordinate system as known input to the eye-tracking system. These parameters, along with the intrinsic ET parameter, participate in the calculation of the human eye's 3D model information (such as the coordinates of the corneal center or the pupil center), ultimately determining the accuracy of the 3D gaze tracking calculation. Therefore, accurate calibration of the intrinsic and extrinsic ET parameters is fundamental to ensuring the accuracy of the eye-tracking algorithm. Large calibration errors can not only reduce the accuracy of eye tracking but may even lead to tracking failure.

[0004] A common ET intrinsic parameter calibration scheme involves capturing multiple checkerboard images at different angles and positions, and then using the Zhang Zhengyou calibration algorithm to solve for the intrinsic parameter matrix and distortion coefficients. However, in eye-tracking modules, the optical axes of the ET camera and the pancake lens assembly have a significant angle, especially in built-in modules where the pancake lens effect is added. This results in a significant difference in the imaging distortion characteristics of the ET camera compared to traditional cameras, exhibiting a significant distortion asymmetry. This asymmetry makes it difficult for the traditional Zhang Zhengyou calibration algorithm to achieve sub-pixel calibration requirements. Furthermore, since the working distance of the ET camera is typically less than 30mm, the traditional Zhang Zhengyou calibration algorithm requires capturing multiple images of the calibration board at different poses within this working distance, which is extremely difficult to implement and results in low calibration efficiency.

[0005] Common ET extrinsic parameter calibration schemes typically involve using a binocular auxiliary camera to directly photograph the LEDs in the eye-tracking module, calculating the LED ring position through triangulation, establishing a ring coordinate system, and finally obtaining the LED extrinsic parameter coordinates in the camera coordinate system. However, with the continuous optimization and thinning of pancake designs, most eye-tracking modules have shifted to semi-integrated or fully integrated designs, embedding the ET camera and / or ET light source into the pancake structure. This makes it easy for the ET light source's emission to be blocked by the structure, making it difficult to emit light beyond the eye-tracking module's ER (interval adaptation distance). Consequently, the binocular auxiliary camera cannot simultaneously view all the built-in LEDs, rendering this calibration method ineffective. Furthermore, for fully integrated eye-tracking modules, the ET light source, like the ET camera, is affected by the pancake lenses, resulting in optical path deflection. Even if the binocular auxiliary camera can directly photograph the LEDs, the calculated ET light source is difficult to keep consistent with the extrinsic parameters used in the algorithm, compromising usability. Utility Model Content

[0006] One advantage of this application is that it provides an eye-tracking module calibration device that is compatible with various eye-tracking layouts, enabling fully automatic calibration of eye-tracking modules and improving the efficiency and accuracy of ET intrinsic and extrinsic parameter calibration.

[0007] Another advantage of this application is that it provides an eye-tracking module calibration device, wherein, in one embodiment of this application, the eye-tracking module calibration device can improve calibration accuracy while reducing the complexity of calibration operations and improving the requirements for the calibration environment.

[0008] Another advantage of this application is that it provides an eye-tracking module calibration device. In one embodiment of this application, the eye-tracking module calibration device is compatible with the calibration of intrinsic and extrinsic parameters under unknown distortion models, which increases the universality of the calibration algorithm for application scenarios and facilitates its widespread application.

[0009] Another advantage of this application is that it provides an eye-tracking module calibration device. In one embodiment of this application, the eye-tracking module calibration device can realize self-verification of calibration results and complete the closed loop of the system calibration process.

[0010] Another advantage of this application is that it provides an eye-tracking module calibration device that does not require a complex structure to achieve the above objectives. Therefore, this application successfully and effectively provides a solution that not only provides a simple eye-tracking module calibration device, but also increases the practicality and reliability of the eye-tracking module calibration device.

[0011] To achieve at least one of the above advantages or other benefits and objectives of this application, this application provides an eye-tracking module calibration device, comprising: a cabinet including a device frame, an electrical base plate fixed to the device frame, a workbench fixed to the device frame and located above the electrical base plate, and an industrial control computer mounted on the electrical base plate; a device fixture mounted on the workbench for fixing the device to be calibrated; a functional calibration plate assembly including a planar calibration plate with a characteristic pattern and a spherical calibration plate with one or more reflective spheres; and a robotic arm assembly mounted in the cabinet and communicatively connected to the industrial control computer, capable of selectively moving the planar calibration plate or the spherical calibration plate in the functional calibration plate assembly to multiple calibration points.

[0012] In one embodiment of this application, the functional label assembly further includes a label fixture for the robotic arm assembly to hold, wherein the planar label and the spherical label are fixed at intervals to the label fixture.

[0013] In one embodiment of this application, the label fixture includes a clamping rod extending upward from the side wall of the spherical label and being held by the robotic arm assembly, and a connecting rod extending downward from the side wall of the spherical label to the planar label.

[0014] In one embodiment of this application, the connecting rod extends obliquely from the spherical marker to the planar marker, and the angle between the plane of the planar marker and the plane of the spherical marker is an obtuse angle.

[0015] In one embodiment of this application, the planar marker is a double-sided checkerboard marker; the double-sided checkerboard marker has an A-side feature pattern arranged opposite to the reflective sphere of the spherical marker and a B-side feature pattern arranged in the same direction as the reflective sphere of the spherical marker.

[0016] In one embodiment of this application, the spherical marker further has a planar feature pattern, which is located on opposite sides of the reflective sphere to form a double-sided marker.

[0017] In one embodiment of this application, the eye-tracking module calibration device further includes a binocular image acquisition device mounted on the workbench and communicatively connected to the industrial control computer; the binocular image acquisition device includes a camera mounting base fixed to the workbench and arranged opposite to the device fixture, and a binocular auxiliary camera mounted on the camera mounting base and communicatively connected to the industrial control computer.

[0018] In one embodiment of this application, the eye-tracking module calibration device further includes an auxiliary light source disposed on the workbench and communicatively connected to the industrial control computer; the auxiliary light source includes a binocular auxiliary fill light mounted on the camera mounting base and communicatively connected to the industrial control computer and an ET auxiliary fill light movably disposed on the workbench and communicatively connected to the industrial control computer.

[0019] In one embodiment of this application, the robotic arm assembly includes a robotic arm body mounted on the workbench and clamping and fixing the functional label assembly, and a robotic arm control cabinet mounted on the electrical base plate and communicatively connected to the robotic arm body and the industrial control computer.

[0020] In one embodiment of this application, the cabinet further includes a device top plate fixedly connected to the device frame, a door assembly installed on the device frame and located between the electrical base plate and the device top plate, an indicator light for displaying device status, and control buttons for starting calibration and resetting calibration; the door assembly includes a front door disposed on the front side of the device frame, a rear door disposed on the rear side of the device frame, and a pair of side doors disposed on the left and right sides of the device frame respectively; the workbench extends forward from the front door to divide the work area inside the device frame and the operation area outside the device frame; the indicator light and the control buttons are both arranged in the operation area of ​​the workbench.

[0021] In one embodiment of this application, the eye-tracking module calibration device further includes an air plug and a pressure regulating valve communicatively connected to the industrial control computer, both of which are installed on the rear door of the device. Attached Figure Description

[0022] Figure 1 This is a perspective view of an eye-tracking module calibration device according to an embodiment of this application;

[0023] Figure 2 A rear-view schematic diagram of an eye-tracking module calibration device according to the above embodiments of this application is shown;

[0024] Figure 3 A schematic diagram showing the open state of the pneumatic sliding door in the eye-tracking module calibration device according to the above embodiments of this application is shown;

[0025] Figure 4 A schematic diagram of the state of the eye-tracking module calibration device according to the above embodiments of this application after the device backdoor is omitted is shown;

[0026] Figure 5 A partial structural schematic diagram of the eye-tracking module calibration device according to the above embodiments of this application is shown;

[0027] Figure 6 A schematic diagram of the structure of the functional calibration plate assembly held by the robotic arm body in the eye-tracking module calibration device according to the above embodiments of this application is shown.

[0028] Explanation of key component symbols:

[0029] 1. Eye-tracking module calibration equipment; 10. Cabinet; 11. Equipment frame; 12. Electrical base plate; 13. Workbench; 131. Working area; 132. Operating area; 14. Industrial computer; 15. Equipment top plate; 16. Door assembly; 161. Front door; 162. Rear door; 163. Side door; 17. Indicator lights; 18. Control buttons; 19. Switch buttons; 20. Equipment fixtures; 30. Functional label assembly; 31. Flat label; 310. Double-sided checkerboard label; 311. Feature pattern on side A; 312, Feature pattern on side B; 32, Spherical marker; 321, Reflective sphere; 322, Planar feature pattern; 33, Marker fixture; 331, Clamping rod; 332, Connecting rod; 40, Robotic arm assembly; 41, Robotic arm body; 42, Robotic arm control cabinet; 50, Binocular image acquisition device; 51, Camera mounting base; 52, Binocular auxiliary camera; 60, Auxiliary light source; 61, Binocular auxiliary fill light; 62, ET auxiliary fill light; 70, Aviation plug; 80, Air pressure regulating valve.

[0030] The above description of the main component symbols, together with the accompanying drawings and specific embodiments, provides a further detailed explanation of this application. Detailed Implementation

[0031] The following description is intended to disclose this application and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of this application defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of this application.

[0032] In the description of this application, it should be understood that terms such as "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this application, it should be noted that, unless otherwise expressly specified and limited, terms such as "connected" or "linked" should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral connection; it can refer to a mechanical connection or an electrical connection; it can refer to a direct connection or an indirect connection through a medium. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0033] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0034] Considering that the ET camera and / or ET light source (such as LED beads) of the pancake eye-tracking module in XR near-eye devices are usually embedded in the pancake lens structure, the LED light emission is easily blocked by the structure and cannot be emitted to areas beyond the module's suitable eye distance range. On the one hand, existing ET intrinsic parameter calibration methods require taking multiple images of the target plate in different poses within the suitable eye distance range, which is extremely difficult to operate and has low calibration efficiency, making it difficult to achieve sub-pixel calibration requirements. On the other hand, existing ET extrinsic parameter calibration methods cannot simultaneously directly capture / co-examine all the built-in LED beads using a binocular auxiliary camera, causing this ET extrinsic parameter calibration scheme to fail. Based on this, this application creatively proposes an eye-tracking module calibration device that is compatible with various eye-tracking layouts, realizes fully automatic calibration of the eye-tracking module, and improves the efficiency and accuracy of ET intrinsic and extrinsic parameter calibration.

[0035] Specifically, refer to the accompanying drawings in the specification of this application. Figures 1 to 6 According to one embodiment of this application, an eye-tracking module calibration device 1 is provided, which may include a cabinet 10, a device fixture 20, a functional calibration plate assembly 30, and a robotic arm assembly 40. The cabinet 10 includes a device frame 11, an electrical base plate 12 fixed to the device frame 11, a workbench 13 fixed to the device frame 11 and located above the electrical base plate 12, and an industrial control computer 14 mounted on the electrical base plate 12 and configured with a calibration algorithm program. The device fixture 20 is mounted on the workbench 13 for fixing the device to be calibrated. The functional calibration plate assembly 30 includes a planar calibration plate 31 with a characteristic pattern and a spherical calibration plate 32 with one or more reflective spheres 321. The robotic arm assembly 40 is mounted on the cabinet 10 and communicatively connected to the industrial control computer 14, and can selectively move the planar calibration plate 31 or the spherical calibration plate 32 to multiple calibration points. It is understood that the device to be calibrated mentioned in this application may be a single eye-tracking module or an XR device that includes two eye-tracking modules, left and right.

[0036] Thus, when the ET intrinsic parameters of the device to be calibrated need to be calibrated, the industrial control computer 14 first controls the robotic arm assembly 40 to move the planar calibration plate 31 to the initial calibration point; then controls the ET camera of the device to be calibrated to acquire the feature pattern image of the planar calibration plate 31 to obtain the intrinsic parameter calibration image at the initial calibration point; then further controls the robotic arm assembly 40 to translate the planar calibration plate 31 to the next calibration point, and further controls the ET camera to acquire the feature pattern image of the planar calibration plate 31 to obtain the intrinsic parameter calibration image at the next calibration point; this process is repeated to obtain multiple intrinsic parameter calibration images at different calibration points; finally, the industrial control computer 14 can call the intrinsic parameter calibration algorithm to process the intrinsic parameter calibration images acquired at multiple calibration points to obtain the calibration optical path table as the ET intrinsic parameters of the device to be calibrated.

[0037] When it is necessary to calibrate the ET extrinsic parameters of the device to be calibrated, the industrial control computer 14 first controls the robotic arm assembly 40 to move the spherical calibration plate 32 to different calibration points in sequence, so that the reflective spheres 321 of the spherical calibration plate 32 are all within the eye-friendly distance range of the device to be calibrated; then, it controls the ET light source of the device to be calibrated to be lit, and controls the ET camera of the device to be calibrated to take pictures of the reflective spheres 321 at different calibration points to obtain multiple extrinsic parameter calibration images at different calibration points; finally, the industrial control computer 14 can call the extrinsic parameter calibration algorithm to process the extrinsic parameter calibration images collected at multiple calibration points to obtain the calibration of the ET light source in the coordinate system of the device to be calibrated as the ET extrinsic parameters of the device to be calibrated.

[0038] It is worth noting that, since the eye-tracking module calibration device 1 of this application only needs to translate the planar target plate 31 during the ET intrinsic parameter calibration process, it does not need to rotate the target plate as in the traditional Zhang's calibration algorithm. This helps to reduce the complexity of the calibration operation and improve the requirements for the calibration environment. Therefore, although the eye-tracking module calibration device 1 of this application adopts the whole-machine calibration mode, the placement of the target plate is not interfered with by the temple structure of the XR head-mounted display, ensuring that the camera's field of view can completely cover the target plate, thus improving the calibration accuracy. At the same time, in application scenarios with large distortion, the eye-tracking module calibration device 1 of this application can achieve higher and more stable calibration accuracy than the Zhang's calibration method, and can better solve the problem of ET camera intrinsic parameter calibration when the distortion model is unknown. In other words, the eye-tracking module calibration device 1 of this application can well support the calibration of ET camera optical paths passing through pancake lenses / diopter lenses, etc., without significantly affecting the calibration accuracy.

[0039] Furthermore, since the same ET light source generates multiple light paths reflected into the ET camera from the reflective spheres 321 at different positions to form light spots, the intersection of the incident light rays (i.e., the position of the ET light source) can be found by tracing the multiple reflected light rays obtained by the ET camera in reverse, thereby achieving ET extrinsic parameter calibration. In other words, during the ET extrinsic parameter calibration process, the eye-tracking module calibration device 1 of this application can simulate the human eye receiving the reflection of the ET light source within the appropriate eye distance range through the reflective spheres 321 of the spherical calibration plate 32, which can avoid the disadvantage of common external auxiliary camera direct shooting of ET light source being easily blocked.

[0040] It should be understood that, as readily apparent from the definition of an optical path table, the core of the ET intrinsic parameter calibration method based on the optical path table lies in how to accurately obtain the 3D point sequence of the "light rays emitted" by each pixel on the ET camera in physical space. Therefore, this application can design different system schemes based on different methods of obtaining the 3D coordinates of the feature points of the target plate, such as: a system scheme based on high-precision structural motion control to provide the physical space 3D coordinates of each feature point on the planar target plate 31; or a system scheme based on binocular stereo vision to estimate the physical space 3D coordinates of each feature point on the planar target plate 31.

[0041] For example, the eye-tracking module calibration device 1 of this application can also adopt a system scheme based on binocular stereo vision to estimate the physical space 3D coordinates of each feature point on the planar target plate 31, so that it is not necessary to ensure that the planar target plate 31 is aligned with the pancake lens and leveled, so as to further reduce the complexity of the calibration operation.

[0042] It is understood that in other examples of this application, the robotic arm assembly 40 in the eye-tracking module calibration device 1 of this application can have a high-precision mechanism design and precise motion control. It can not only accurately move the planar target plate 31 to the initial calibration point, ensuring that the center of the planar target plate 31 is coaxially aligned with the pancake lens of the XR head-mounted display device, and that the plane of the planar target plate 31 is parallel to the end face of the pancake lens; but also accurately translate the planar target plate 31 to other calibration points, ensuring that the plane of the planar target plate 31 is parallel to the end face of the pancake lens, so as to accurately give the 3D physical coordinates of each feature point on the planar target plate 31 in the coordinate system of the device to be calibrated by relying on the high-precision mechanism design and precise motion control.

[0043] Specifically, such as Figure 3 and Figure 5As shown, the eye-tracking module calibration device 1 of this application may further include a binocular image acquisition device 50, which is installed on the workbench 13 and communicatively connected to the industrial control computer 14; the industrial control computer 14 can control the binocular image acquisition device 50 to acquire feature pattern images of the planar target plate 31 at different calibration points, so as to calculate the 3D coordinates of the feature points of the target plate at different calibration points.

[0044] Optionally, such as Figure 5 As shown, the binocular image acquisition device 50 includes a camera mounting base 51 fixed to the worktable 13 and a binocular auxiliary camera 52 mounted on the camera mounting base 51 and communicatively connected to the industrial control computer 14; the camera mounting base 51 is arranged opposite to the device clamp 20 so that the robotic arm assembly 40 can move the planar target plate 31 to a position between the binocular auxiliary camera 52 and the device to be calibrated, ensuring that the planar target plate 31 can be within the field of view of the binocular auxiliary camera 52 and the ET camera.

[0045] Optionally, such as Figure 5 and Figure 6 As shown, the planar marker 31 is implemented as a double-sided checkerboard marker 310, which has an A-side feature pattern 311 and a B-side feature pattern 312 arranged opposite to each other. Thus, when the planar marker 31 is at a calibration point, the A-side feature pattern 311 of the double-sided checkerboard marker 310 is completely within the field of view of the binocular auxiliary camera 52 in the binocular image acquisition device 50, and the B-side feature pattern 312 of the double-sided checkerboard marker 310 covers the effective field of view of the ET camera in the device to be calibrated. This allows the binocular auxiliary camera 52 of the binocular image acquisition device 50 to acquire the A-side feature pattern image, and the ET camera in the device to be calibrated to acquire the B-side feature pattern image as an internal parameter calibration image. It is understandable that, since the thickness of the double-sided checkerboard marking plate 310 is known, the positional relationship between the feature points on side B and the feature points on side A is known. Therefore, the industrial control computer 14 of this application can first estimate the 3D coordinates of the feature points on side A through the binocular camera triangulation algorithm, and then convert them into the 3D coordinates of the feature points on side B based on the thickness of the double-sided checkerboard marking plate 310.

[0046] It is worth noting that in other examples of this application, the feature pattern of the planar label 31 may also be implemented as, but is not limited to, a checkerboard pattern with dot markings, a dotted label pattern, a QR code pattern, or a Charuco label pattern, etc.

[0047] Furthermore, in other examples of this application, the camera mounting base 51 and the device clamp 20 can also be arranged side by side, so that the robotic arm assembly 40 can move the planar target 31 to the same side of the binocular auxiliary camera 52 and the device to be calibrated, while still ensuring that the planar target 31 is within the field of view of the binocular auxiliary camera 52 and the ET camera. In this case, the planar target 31 can be implemented as a single-sided checkerboard target, so that the industrial control computer 14 of this application can estimate the 3D coordinates of each feature point on the planar target 31 through the binocular camera triangulation algorithm.

[0048] According to the above embodiments of this application, in order to ensure that the feature points on the planar marker 31 are clear and distinct, such as Figure 5 As shown, the eye-tracking module calibration device 1 of this application further includes an auxiliary light source 60 disposed on the workbench 13 and communicatively connected to the industrial control computer 14, which is used to light up or turn off under the control of the industrial control computer 14 so as to make the brightness of the acquired image uniform.

[0049] Optionally, such as Figure 5 As shown, the auxiliary light source 60 includes a binocular auxiliary fill light 61, which is mounted on the camera mounting base 51 and communicatively connected to the industrial control computer 14. The industrial control computer 14 is used to configure the fill light parameters of the binocular auxiliary fill light 61 according to the sampling point information of the current target board, so as to make the brightness of the image acquired by the binocular auxiliary camera 52 uniform.

[0050] Optionally, such as Figure 5 As shown, the auxiliary light source 60 also includes an ET auxiliary fill light 62, which is movably mounted on the workbench 13 and communicatively connected to the industrial control computer 14. The industrial control computer 14 is used to control the movement of the ET auxiliary fill light 62 to move closer to or further away from the equipment fixture 20, and to configure the fill light parameters of the ET auxiliary fill light 62 according to the sampling point information of the current target plate, so as to make the brightness of the image acquired by the ET camera uniform.

[0051] Optionally, such as Figure 5 As shown, the ET auxiliary light 62 is implemented as a ring light strip, and the lighting of each LED in the ring light strip can be controlled independently to solve the problem of poor image uniformity when the ET camera is built in.

[0052] It is worth noting that, such as Figure 5 and Figure 6As shown, the functional marker assembly 30 may further include a marker fixture 33 for the robotic arm assembly 40 to hold; the planar marker 31 and the spherical marker 32 are fixed at intervals to the marker fixture 33, so that the robotic arm assembly 40 can move the planar marker 31 and the spherical marker 32 synchronously, ensuring that the relative positional relationship between the feature points on the planar marker 31 and the reflective sphere 321 of the spherical marker 32 remains constant and is known. In this way, when performing ET extrinsic calibration, it is only necessary to use the binocular image acquisition device 50 to photograph the planar marker 31, and the position of the reflective sphere 321 of the spherical marker 32 can be estimated visually, without having to ensure high-precision alignment and leveling of the spherical marker 32 with the calibration device pancake.

[0053] For example, such as Figure 6 As shown, the label tooling 33 may include a clamping rod 331 extending upward from the side wall of the spherical label 32 and being held by the robotic arm assembly 40, and a connecting rod 332 extending downward from the side wall of the spherical label 32 to the planar label 31, such that the planar label 31 is located below the spherical label 32.

[0054] Optionally, such as Figure 6 As shown, the connecting rod 332 extends obliquely from the spherical marker plate 32 to the planar marker plate 31, and the angle between the plane of the planar marker plate 31 and the plane of the spherical marker plate 32 is an obtuse angle. Preferably, the angle between the plane of the planar marker plate 31 and the plane of the spherical marker plate 32 is between 100° and 120°. More preferably, the angle between the plane of the planar marker plate 31 and the plane of the spherical marker plate 32 is equal to 110°. Thus, as... Figure 5 As shown, when the spherical target plate 32 is at the calibration point, the connecting rod 332 is tilted towards the binocular image acquisition device 50 to ensure that when the spherical target plate 32 is within the field of view of the ET camera, the planar target plate 31 is completely within the field of view of the binocular auxiliary camera 52. It is understood that the angle between the target plane of the planar target plate 31 and the target plane of the spherical target plate 32 can also be manually adjusted as needed to meet shooting requirements.

[0055] Optionally, such as Figure 5 and Figure 6 As shown, the feature pattern 311 on the A side of the planar marker 31 is arranged opposite to the reflective sphere 321 of the spherical marker 32, and the feature pattern 312 on the B side of the planar marker 31 is arranged in the same direction as the reflective sphere 321 of the spherical marker 32.

[0056] It is worth mentioning that in other examples of this application, the functional label assembly 30 can also use a label box fixed to the workbench 13 to replace the label fixture 33; the planar label 31 and the spherical label 32 can be placed inside the label box, that is, the planar label 31 and the spherical label 32 are relatively independent, and the robotic arm assembly 40 can selectively pick up the planar label 31 or the spherical label 32 for calibration as needed.

[0057] Furthermore, in the above embodiments of this application, such as Figure 5 and Figure 6 As shown, the spherical calibration plate 32 can further have a planar feature pattern 322, which is located on opposite sides of the reflective sphere 321 to form a double-sided calibration plate. Since the relative positional relationship between the planar feature pattern 322 and the reflective sphere 321 of the spherical calibration plate 32 is fixed and known, during ET extrinsic parameter calibration, it is only necessary to use the binocular image acquisition device 50 to capture the planar feature pattern 322 of the spherical calibration plate 32 to visually estimate the position of the reflective sphere 321 of the spherical calibration plate 32, without needing to ensure high-precision alignment and leveling of the spherical calibration plate 32 with the calibration equipment.

[0058] It is worth noting that after completing the ET intrinsic parameter calibration and / or ET extrinsic parameter calibration, the eye-tracking module calibration device 1 of this application can also realize the fully automated verification process to complete the closed loop of the automatic calibration process and ensure the accuracy of the calibration results.

[0059] For example, when automating the verification of ET intrinsic parameter calibration results, the industrial control computer 14 first controls the robotic arm assembly 40 to move the planar target plate 31 or the spherical target plate 32 to the verification point, and then controls the ET camera to acquire the image of the planar target plate 31 or the spherical target plate 32 to obtain the verification image at the verification point; finally, based on the calibration optical path table and the design value of the planar target plate 31 or the spherical target plate 32, the intrinsic parameter verification algorithm is called to execute the verification algorithm on the verification image to verify the calibration optical path table.

[0060] When automating the verification of the ET extrinsic parameter calibration results, the industrial control computer 14 first controls the robotic arm assembly 40 to move the spherical target plate 32 to the verification point, and then controls the ET camera to acquire the image of the spherical target plate 32 to obtain the verification spot image at the verification point; finally, the verified ET intrinsic parameters and the calibrated ET extrinsic parameters are used to call the extrinsic parameter verification algorithm to execute the verification algorithm on the verification spot image to verify the accuracy of the ET extrinsic parameter calibration results.

[0061] According to the above embodiments of this application, as Figure 4 and Figure 5As shown, the robotic arm assembly 40 may include a robotic arm body 41 mounted on the workbench 13 and clamping and fixing the functional calibration plate assembly 30, and a robotic arm control cabinet 42 mounted on the electrical base plate 12 and communicatively connected to the robotic arm body 41 and the industrial control computer 14. The robotic arm control cabinet 42, under the control of the industrial control computer 14, controls the robotic arm body 41 to move the planar calibration plate 31 or the spherical calibration plate 32 in the functional calibration plate assembly 30 to different calibration points. It is understood that the industrial control computer mentioned in this application can be used for multi-camera control, data acquisition, storage, command transmission, data processing, and as a carrier of software algorithms, etc.

[0062] In addition, such as Figure 1 and Figure 2 As shown, the cabinet 10 of the eye-tracking module calibration device 1 of this application may further include a device top plate 15 fixedly connected to the device frame 11 and a door assembly 16 installed on the device frame 11 and located between the electrical base plate 12 and the device top plate 15, so that the space between the electrical base plate 12 and the device top plate 15 can be selectively closed through the door assembly 16.

[0063] For example, such as Figures 1 to 3 As shown, the door assembly 16 includes a front door 161 located on the front side of the equipment frame 11, a rear door 162 located on the rear side of the equipment frame 11, and a pair of side doors 163 located on the left and right sides of the equipment frame 11, respectively. The workbench 13 extends forward from the front door 161 to divide the work area 131 within the equipment frame 11 and the operation area 132 outside the equipment frame 11. The equipment clamp 20, the robotic arm body 41, and the auxiliary light source 60 are arranged in the work area 131 of the workbench 13 to prevent external environmental interference with the calibration process.

[0064] Optionally, such as Figure 1 and Figure 3 As shown, the equipment front door 161 located between the workbench 13 and the equipment top plate 15 is implemented as a pneumatic sliding door; the equipment front door 161 located between the workbench 13 and the electrical base plate 12 is implemented as a swing door.

[0065] Optionally, such as Figure 1 and Figure 3 As shown, the cabinet 10 of the eye-tracking module calibration device 1 of this application also includes an indicator light 17 for displaying the device status and a control button 18 for starting calibration and resetting calibration; the indicator light 17 and the control button 18 are both arranged in the working area 131 of the workbench 13 for the operator to operate.

[0066] Optionally, such as Figure 1As shown, the cabinet 10 of the eye-tracking module calibration device 1 of this application is used to turn on or off the switch button 19 of the relevant electrical equipment in the device. The switch button 19 is located on the front door 161 of the device. It is understood that the switch button 19 mentioned in this application may, but is not limited to, be implemented as a rotary switch.

[0067] It is worth noting that, such as Figure 2 As shown, the eye-tracking module calibration device 1 of this application may further include an aviation plug 70 communicatively connected to the industrial control computer 14. The aviation plug 70 is installed on the rear door 162 of the device and is used to communicate between the industrial control computer 14 and an external interface.

[0068] In addition, such as Figure 2 As shown, the eye-tracking module calibration device 1 of this application may also include an air pressure regulating valve 80 disposed on the rear door 162 of the device, for regulating the internal air pressure of the cabinet 10 to ensure that the internal air pressure of the cabinet 10 can meet the calibration requirements.

[0069] In this way, during the calibration process, after the device to be calibrated is fixed to the device clamp 20 from the pneumatic sliding door, the control button 18 is activated: the pneumatic sliding door closes automatically, the robotic arm assembly 40 automatically moves the functional calibration plate assembly 30 so that the planar calibration plate 31 moves to different calibration points, the ET auxiliary light is turned on, and the ET camera is controlled to acquire feature pattern images of the planar calibration plate 31; after the feature pattern images are acquired, the ET auxiliary light is moved away to prevent obstruction of the device to be calibrated, and the robotic arm assembly 40 is controlled to automatically move the functional calibration plate assembly 30 so that the spherical calibration plate 32 moves to different calibration points, and the ET camera is controlled to acquire spot images when the ET light source is lit; at the same time, the feature pattern images and spot images are automatically processed to calculate the ET intrinsic and extrinsic parameters, and the device to be calibrated is judged as qualified based on the calculation results.

[0070] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0071] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are quite specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application.

Claims

1. An eye movement module calibration apparatus, characterized in that, include: The cabinet includes an equipment frame, an electrical base plate fixed to the equipment frame, a workbench fixed to the equipment frame and located above the electrical base plate, and an industrial control computer mounted on the electrical base plate. A device fixture is mounted on the workbench to fix the device to be calibrated; A functional signboard assembly, comprising a planar signboard with a characteristic pattern and a spherical signboard with one or more reflective spheres; as well as The robotic arm assembly, installed in the cabinet and communicatively connected to the industrial computer, can selectively move the planar or spherical marker in the functional marker assembly to multiple calibration points.

2. The eye movement module calibration device of claim 1, wherein, The functional label assembly also includes a label fixture for the robotic arm assembly to hold, wherein the planar label and the spherical label are fixed to the label fixture at intervals.

3. The eye movement module calibration device of claim 2, wherein, The marking fixture includes a clamping rod extending upward from the side wall of the spherical marking plate and being held by the robotic arm assembly, and a connecting rod extending downward from the side wall of the spherical marking plate to the planar marking plate.

4. The eye movement module calibration device of claim 3, wherein, The connecting rod extends obliquely from the spherical marker to the planar marker, and the angle between the plane of the planar marker and the plane of the spherical marker is an obtuse angle.

5. The eye movement module calibration device of claim 4, wherein, The planar marker is a double-sided checkerboard marker; the double-sided checkerboard marker has an A-side feature pattern arranged opposite to the reflective sphere of the spherical marker and a B-side feature pattern arranged in the same direction as the reflective sphere of the spherical marker.

6. The eye movement module calibration device of claim 1, wherein, The spherical marker also has a planar feature pattern, which is located on opposite sides of the reflective sphere to form a double-sided marker.

7. The eye movement module calibration device according to any one of claims 1 to 6, wherein, The eye-tracking module calibration device also includes a binocular image acquisition device mounted on the workbench and communicatively connected to the industrial control computer; the binocular image acquisition device includes a camera mounting base fixed to the workbench and arranged opposite to the device fixture, and a binocular auxiliary camera mounted on the camera mounting base and communicatively connected to the industrial control computer.

8. The eye movement module calibration device of claim 7, wherein, The eye-tracking module calibration device further includes an auxiliary light source disposed on the workbench and communicatively connected to the industrial control computer; the auxiliary light source includes a binocular auxiliary fill light mounted on the camera mounting base and communicatively connected to the industrial control computer and an ET auxiliary fill light movably disposed on the workbench and communicatively connected to the industrial control computer.

9. The eye movement module calibration device of any one of claims 1 to 6, wherein, The robotic arm assembly includes a robotic arm body mounted on the workbench and clamping and fixing the functional label assembly, and a robotic arm control cabinet mounted on the electrical base plate and communicatively connected to the robotic arm body and the industrial computer.

10. The eye movement module calibration device of any one of claims 1 to 6, wherein, The cabinet also includes a top plate fixedly connected to the equipment frame, a door assembly installed on the equipment frame and located between the electrical base plate and the top plate, indicator lights for displaying equipment status, and control buttons for starting and resetting calibration; the door assembly includes a front door located on the front side of the equipment frame, a rear door located on the rear side of the equipment frame, and a pair of side doors respectively located on the left and right sides of the equipment frame; the workbench extends forward from the front door to divide the work area inside the equipment frame and the operating area outside the equipment frame; the indicator lights and the control buttons are both located in the operating area of ​​the workbench; the eye-tracking module calibration device also includes an air plug and a pressure regulating valve communicatively connected to the industrial control computer, and the air plug and the pressure regulating valve are both installed on the rear door.

Images