Eye tracking imaging system and method

Through the combination of telephoto cameras and motion platforms, combined with wide-angle cameras and deep convolutional network models, the problem of difficulty in obtaining micro feature information and large field of view images and off-target eye imaging in traditional eye imaging technology is solved, and the problem of off-target eye imaging is realized in a dynamic environment high resolution eye tracking imaging.

CN114376505BActive Publication Date: 2024-12-03RUIXIN ZHIPU (SHENZHEN) TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111669580.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-31
Publication Date
2024-12-03
Estimated Expiration
2041-12-31

AI Technical Summary

Technical Problem

Traditional eye imaging technology is difficult to obtain high-resolution micro feature information and large field of view images at the same time, and it is easy for the eye imaging to off-target when the user moves, resulting in poor user experience.

Method used

Using a telephoto camera combined with a motion platform, the telephoto camera focal length is fixed through the first and second motion components, adjust the lens pitch angle and roll angle, combine with a wide-angle camera to acquire facial images in real time, and calculate and adjust the length using a deep convolutional network model. The optical axis angle of the focus camera realizes motion control of the eye tracking imaging system.

Benefits of technology

It realizes real-time acquisition of clear eye micro-feature images in a dynamic large field of view environment, avoids off-target imaging of eyeballs, and improves the stability of target positioning and image clarity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114376505B_ABST
    Figure CN114376505B_ABST
Patent Text Reader

Abstract

The present invention provides an eye tracking imaging system and method, the eye tracking imaging system comprises: a telephoto camera for acquiring an eye micro-feature image in real time; a motion platform, comprising a first motion component and a second motion component, the first motion component being drivingly connected to the telephoto camera, for keeping the telephoto camera focal length fixed and driving the telephoto camera to rotate about a first direction to adjust the telephoto camera lens pitch angle; the second motion component being drivingly connected to the telephoto camera, for keeping the telephoto camera focal length fixed and driving the telephoto camera to rotate about a second direction to adjust the telephoto camera lens roll angle. The present invention enables the eye tracking imaging system to locate the target without missing the target, and the obtained target image has high definition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of tracking imaging technology, and in particular to an eye tracking imaging system and method. Background Art

[0002] With the continuous development of science and technology, personal biometric information identification (PBIC) based on the extraction of eyeball features has been applied to many aspects of production and life, such as industrial production, security protection, and media entertainment. The extraction of image eyeball feature information is the premise and basis for the recognition and analysis of personal behavior patterns, and then the behavior and consciousness of people can be analyzed. In the process of eyeball imaging, traditional wide-angle cameras can stably obtain eyeball imaging information in real time, but due to its imaging resolution, its micro-feature information is lost. Telephoto cameras can obtain micro-feature information of eyeball imaging in real time, but their imaging field of view is small. During the imaging process, due to human movement, eyeball features are easily off-target and difficult to image. Therefore, the recognition and analysis of personal behavior patterns based on eyeball features faces many difficulties.

[0003] At present, the commonly used method is to aim the imaging device at the eyeball at a close distance to obtain the micro-feature information of the eyeball. This method has great limitations. The eyeball to be photographed must be kept still and can be photographed at a close distance. If the eyeball shooting distance is too far, it is difficult to obtain high-resolution micro-feature information of the eyeball, and even pixel blurring and other problems may occur. If the eyeball shooting distance is too close, it is easy to cause problems such as difficulty in obtaining complete micro-feature information of the eyeball. In addition, if the user moves slightly, the eyeball imaging is likely to be off-target, making it difficult to obtain micro-feature information of the eyeball. This type of method has many requirements for the monitored person, and the user experience is extremely poor, making it difficult to put into actual use; another method is to use a zoom camera. This method first uses a short focus to search for the approximate spatial position of the target, and then uses a long focus. If a close-up eye shot is not achieved but a wide-angle shot is selected, it is impossible to obtain clear eyeball information at all, and only low-pixel eyeball image information under a wide angle can be obtained. There are disadvantages such as incomplete image information and pixel blur. Summary of the invention

[0004] In view of the above problems, the present invention provides an eye tracking imaging system and method, which effectively solves at least one of the above problems.

[0005] In order to solve the above problems, the present invention provides an eye tracking imaging system, comprising:

[0006] A telephoto camera for acquiring images of eye micro-features in real time;

[0007] A motion platform, comprising a first motion component and a second motion component, wherein the first motion component is drivingly connected to the telephoto camera, and is used to keep the focal length of the telephoto camera fixed and drive the telephoto camera to rotate around a first direction to adjust the pitch angle of the telephoto camera lens; the second motion component is drivingly connected to the telephoto camera, and is used to keep the focal length of the telephoto camera fixed and drive the telephoto camera to rotate around a second direction to adjust the roll angle of the telephoto camera lens.

[0008] Optionally, the motion platform also includes a first mounting platform, and the telephoto camera is arranged on the first mounting platform; the first motion component includes a first driving member, and the first driving member is drivingly connected to the first mounting platform for driving the first mounting platform to rotate around the first direction.

[0009] Optionally, the motion platform also includes a second mounting platform, and the second motion component is arranged on the second mounting platform; the second mounting platform is movably arranged on the first mounting platform; the telephoto camera is arranged on the first mounting platform through the second mounting platform; the second motion component includes a second driving member, and the second driving member is drivingly connected to the second mounting platform for driving the second mounting platform to rotate around a second direction.

[0010] Optionally, an arc groove is provided on the first mounting platform, and a guide block is provided on the end surface of the second mounting platform facing the first mounting platform, the guide block is slidably set in the arc groove, and the line connecting the center of the arc groove and the center point of the lens of the telephoto camera is respectively perpendicular to the radius of the arc groove and the central axis of the lens of the telephoto camera.

[0011] Optionally, the second motion assembly further comprises a telescopic member capable of realizing telescoping, one end of the telescopic member is drivingly connected to the second driving member, and the other end of the telescopic member is connected to the second carrying platform;

[0012] When the second driving member drives the second carrying platform through the telescopic member, the telescopic member is adapted to be extended or shortened to allow the guide block to slide in the arc groove.

[0013] Optionally, the eye tracking imaging system also includes a fixed seat, a mounting frame and a wide-angle camera, the wide-angle camera is used to obtain real-time facial images of each person in a dynamic and large-field-of-view multi-information environment, the fixed seat is provided with a mounting groove, the mounting frame and the motion platform are both arranged in the mounting groove, and the wide-angle camera is installed on the mounting frame; the side wall of the fixed seat is provided with an imaging port communicating with the mounting groove, and the optical path difference between the wide-angle camera and the telephoto camera to the imaging port is zero.

[0014] Optionally, the eye tracking imaging system further includes a control component and a cover, wherein the control component is disposed on the fixed base, and the control component is electrically connected to the wide-angle camera, the telephoto camera and the motion platform; the cover is disposed on the fixed base, and the control component is located inside the cover.

[0015] Another object of the present invention is to provide an eye tracking imaging method, which uses the eye tracking imaging system as described above, comprising:

[0016] Real-time acquisition of facial images of each person in dynamic and large-field-of-view multi-information environments;

[0017] According to the facial image, a roll angle and a pitch angle between the eyeball to be monitored and the optical axis of the telephoto camera of the eye tracking imaging system are obtained;

[0018] Based on a preset conversion rule, convert the roll angle and the pitch angle into a motion instruction;

[0019] Based on the motion instruction, the motion platform of the eye tracking imaging system is controlled to move.

[0020] Optionally, obtaining the roll angle and pitch angle of the eyeball to be monitored and the optical axis of the telephoto camera of the eye tracking imaging system according to the facial image includes:

[0021] Extracting facial feature information from the facial image based on face matching and an improved deep convolutional network model to obtain a first real-time coordinate of a target image;

[0022] Obtaining a second real-time coordinate of the optical axis of the telephoto camera in the facial image according to servo feedback of the first driving member and the second driving member of the eye tracking imaging system;

[0023] According to the first real-time coordinate and the second real-time coordinate, the roll angle and the pitch angle of the eyeball to be monitored and the optical axis of the telephoto camera are calculated.

[0024] Optionally, converting the roll angle and the pitch angle into a motion instruction based on a preset conversion rule includes:

[0025] According to the mechanical principle and the corresponding relationship between the swing angle of the telephoto camera, the roll angle and the pitch angle are converted into a transfer function for controlling the roll angle and the pitch angle of the telephoto camera, and a motion instruction for controlling the motion platform is obtained through the transfer function.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] When the eye tracking imaging system is working, the telephoto camera is used to obtain the eye micro-feature image in real time, and the obtained target image has high definition; at the same time, the first motion component of the motion platform is connected to the telephoto camera driving, which can keep the telephoto camera focal length fixed and drive the telephoto camera to rotate around a first direction to adjust the pitch angle of the telephoto camera lens; the second motion component is connected to the telephoto camera driving, which can keep the telephoto camera focal length fixed and drive the telephoto camera to rotate around a second direction to adjust the roll angle of the telephoto camera lens. After adjusting the roll angle and pitch angle of the telephoto camera, the telephoto camera focuses and obtains the eye micro-feature image in real time. Not only is the obtained target image high in definition, but also when the telephoto camera obtains the eye micro-feature image, the positioning target is prevented from missing the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is an exploded schematic diagram of the eye tracking imaging system of the present invention;

[0029] Figure 2 This is a schematic structural diagram of an implementation mode of the motion platform of the present invention;

[0030] Figure 3 This is a schematic structural diagram of an implementation mode of the first carrying platform in the present invention;

[0031] Figure 4 It is a schematic diagram of the assembly of the eye tracking imaging system of the present invention;

[0032] Figure 5 This is a structural schematic diagram of an implementation mode of the fixing seat in the present invention;

[0033] Figure 6 A flowchart of an implementation method of the eye tracking imaging method of the present invention;

[0034] Figure 7 The present invention is a flowchart of an embodiment of the eye tracking imaging method.

[0035] Description of reference numerals:

[0036] 1. Wide-angle camera; 2. Telephoto camera; 3. Moving platform; 31. First driving member; 32. First carrying platform; 321. Connecting member; 322. Mounting plate; 323. Connecting edge; 324. Arc groove; 3211. Card seat; 3212. First sliding rod; 3213. Connecting buckle; 33. First limiting member; 34. Second carrying platform; 341. Connecting column; 35. Second driving member; 36. Telescopic member; 361. Sliding sleeve; 362. Moving rod; 37. Second limiting member; 4. Fixed seat; 41. Imaging port; 42. Mounting slot; 5. Control assembly; 51. Strawberry pie; 52. Switch; 6. Cover; 7. Mounting frame; 8. Lens assembly; 81. Half-reflective half-mirror; 82. Reflector. DETAILED DESCRIPTION

[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0039] The Z-axis in the drawings represents the vertical direction, that is, the up and down position, and the positive direction of the Z-axis (that is, the direction of the arrow of the Z-axis) represents the top, and the reverse direction of the Z-axis represents the bottom; the X-axis in the drawings represents the horizontal direction, and is designated as the left and right positions, and the positive direction of the X-axis represents the left side, and the reverse direction of the X-axis represents the right side; the Y-axis in the drawings represents the front and back direction, and is designated as the front and back positions, and the positive direction of the Y-axis represents the front side, and the reverse direction of the Y-axis represents the back side; it should be noted that the aforementioned Z-axis, X-axis and Y-axis are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0040] like Figure 1 , 2 As shown, the eye tracking imaging system of an embodiment of the present invention includes a telephoto camera 2 and a motion platform 3, wherein the telephoto camera 2 is used to obtain eye micro-feature images in real time, and the motion platform 3 includes a first motion component and a second motion component, the first motion component is drivingly connected to the telephoto camera 2, and is used to keep the focal length of the telephoto camera 2 fixed and drive the telephoto camera 2 to rotate around a first direction to adjust the pitch angle of the telephoto camera 2; the second motion component is drivingly connected to the telephoto camera 2, and is used to keep the focal length of the telephoto camera 2 fixed and drive the telephoto camera 2 to rotate around a second direction to adjust the roll angle of the telephoto camera 2.

[0041] In this embodiment, the first direction refers to the rotation direction of the first mounting platform 2 relative to the fixing base 4 (described later), that is, Figure 1The first direction is the direction where the X-axis is located. The second direction refers to the axial direction of the arc groove 324 (described later). The telephoto camera 2 has an automatic focus function. After adopting the eye tracking imaging system of this embodiment, when working, the first motion component of the motion platform 3 is connected to the telephoto camera 2 for driving and maintaining the focal length of the telephoto camera 2 and driving the telephoto camera 2 to rotate around the first direction to adjust the pitch angle of the lens of the telephoto camera 2; the second motion component is connected to the telephoto camera 2 for driving and maintaining the focal length of the telephoto camera 2 and driving the telephoto camera 2 to rotate around the second direction to adjust the roll angle of the lens of the telephoto camera 2. After adjusting the roll angle and pitch angle of the telephoto camera 2, the telephoto camera 2 focuses and obtains the eye micro-feature image in real time. Not only is the target image obtained with high clarity, but also when the telephoto camera obtains the eye micro-feature image, it prompts the eye tracking imaging system to locate the target without missing the target.

[0042] Optionally, the motion platform 3 also includes a first mounting platform 32, and the telephoto camera 2 is arranged on the first mounting platform 32; the first motion component includes a first driving member 31, and the first driving member 31 is drivingly connected to the first mounting platform 32, and is used to drive the first mounting platform 32 to rotate around the first direction.

[0043] In this embodiment, the first driving member 31 includes a first motor or a first cylinder, and the first carrying platform 32 is rotatably connected to the fixed seat 4 (described later); when the first driving member 31 includes a first motor, the first carrying platform 32 is threadedly connected to the rotating shaft of the first motor through a connecting member 321. After the first motor is powered on, the rotating shaft of the first motor rotates. Due to the existence of the connecting member 321, one end of the first carrying platform 32 is driven by the first motor to rise or fall, and finally the first carrying platform 32 is rotated around the rotating connection between the first carrying platform 32 and the fixed seat 4. Similarly, when the first driving member 31 includes a first cylinder, when the first cylinder is working, one end of the first carrying platform 32 is driven by the first cylinder to rise or fall, and finally the first carrying platform 32 is rotated around the rotating connection between the first carrying platform 32 and the fixed seat 4.

[0044] With such arrangement, under the action of the first driving member 31, the first mounting platform 32 can be rotated around the first direction. Since the telephoto camera 2 is arranged on the first mounting platform 32, the telephoto camera 2 can rotate around the first direction following the first mounting platform 32, thereby adjusting the pitch angle of the telephoto camera 2.

[0045] Optionally, the eye tracking imaging system also includes a first limiting member 33 and a connecting member 321, the first driving member 31 includes a first motor, one end of the connecting member 321 is slidably connected to the first carrying platform 32, and the other end of the connecting member 321 is provided with a movably connected connecting buckle 3213, the rotating shaft of the first motor is threadedly connected to the connecting buckle 3213, and the first limiting member 33 is located on the moving path of the connecting buckle 3213 to limit the connecting buckle 3213 from being separated from the rotating shaft of the first motor.

[0046] Specifically, the first motor is arranged on the side of the first loading platform 32, and a card seat 3211 is provided on the first loading platform 32. The connecting member 321 includes a first slide bar 3212 slidably connected to the card seat 3211. Specifically, a sliding groove is provided on the card seat 3211, one end of the first slide bar 3212 is slidably connected to the sliding groove, and the other end of the first slide bar 3212 is provided with a movably connected connecting buckle 3213, and the rotating shaft of the first motor is threadedly connected to the connecting buckle 3213. For example, an external thread is provided on the rotating shaft of the first motor, and the connecting buckle 3213 is provided with a threaded channel. The connecting buckle 3213 is threadedly connected to the shaft of the first motor through a threaded channel. When the shaft of the first motor drives the connecting buckle 3213, the first slide bar 3212 and the connecting buckle 3213 rotate relative to each other. The first limiter 33 is located on the moving path of the connecting buckle 3213 to limit the connecting buckle 3213 from being separated from the shaft of the first motor. Specifically, the first limiter 33 is located near the free end of the shaft of the first motor. When the connecting buckle 3213 moves to the free end of the shaft of the first motor, it is blocked by the first limiter 33, so the connecting buckle 3213 will not be separated from the shaft of the first motor. Figure 2 As shown, the first limit member 33 is Z-shaped, one end of the first limit member 33 is fixed on the fixing seat 4 (introduced later), and the other end of the first limit member 33 is located above the rotating shaft of the first motor. There is a gap between the part of the first limit member 33 located above the rotating shaft of the first motor and the rotating shaft of the first motor, and the gap is smaller than the depth of the threaded hole of the first slide bar 3212, so the first limit member 33 will not hinder the movement of the rotating shaft of the first motor.

[0047] With such arrangement, under the action of the first limiting member 33 , the travel of the connecting member 321 can be limited, thereby preventing the connecting member 321 from being separated from the rotating shaft of the first motor.

[0048] Optionally, the motion platform 3 also includes a second mounting platform 34, and a second motion component is arranged on the second mounting platform 34; the second mounting platform 34 is movably arranged on the first mounting platform 32; the telephoto camera 2 is arranged on the first mounting platform 32 through the second mounting platform 34; the second motion component includes a second driving member 35, and the second driving member 35 is drivingly connected to the second mounting platform 34, and is used to drive the second mounting platform 34 to rotate around a second direction.

[0049] Specifically, the second motion assembly is disposed on the second mounting platform 34. Specifically, a mounting plate 322 is disposed on the second mounting platform 34, and the second motion assembly is fixed to the mounting plate 322 by screws. The second mounting platform 34 is movably disposed on the first mounting platform 32, and the second mounting platform 34 can rotate around the second direction; the telephoto camera 2 is disposed on the first mounting platform 32 through the second mounting platform 34. When the second mounting platform 34 rotates around the second direction on the first mounting platform 32, the telephoto camera 2 also rotates around the second direction, thereby adjusting the roll angle of the lens of the telephoto camera 2.

[0050] In this embodiment, the second driving member 35 includes a second motor or a second cylinder, and the second motor or the second cylinder is fixed on the mounting plate 322; when the second driving member 35 includes a second motor, the rotating shaft of the second motor is threadedly connected to the second carrying platform 34. After the second motor is powered on, the rotating shaft of the second motor rotates, and the part of the second carrying platform 34 connected to the rotating shaft of the second motor moves along the axial direction of the rotating shaft, thereby causing the second carrying platform 34 to move relative to the first carrying platform 32. When the second driving member 35 includes a second cylinder, the telescopic rod of the second cylinder is connected to the second carrying platform 34. When the second cylinder is working, the part of the second carrying platform 34 connected to the telescopic rod of the second cylinder moves with the telescopic rod, thereby causing the second carrying platform 34 to move relative to the first carrying platform 32.

[0051] With such arrangement, under the action of the second driving member 35, the second carrying platform 34 can be moved on the first carrying platform 32. Since the telephoto camera 2 is arranged on the first carrying platform 32 through the second carrying platform 34, the telephoto camera 2 can move on the first carrying platform 32 following the second carrying platform 34, thereby adjusting the position of the telephoto camera 2.

[0052] Optionally, an arc groove 324 is provided on the first mounting platform 32, and a guide block is provided on the end surface of the second mounting platform 34 facing the first mounting platform 32, and the guide block is slidably set in the arc groove 324, and the line connecting the center of the arc groove 324 and the center point of the lens of the telephoto camera 2 is respectively perpendicular to the radius of the arc groove 324 and the central axis of the lens of the telephoto camera 2.

[0053] Specifically, Figure 3 As shown, the first mounting platform 32 is provided with an arc groove 324 along the thickness direction (the direction where the Z axis is located), and the second mounting platform 34 is provided with a guide block on the end surface facing the first mounting platform 32. The guide block is arc-shaped and is slidably set in the arc groove 324. The line connecting the center of the arc groove 324 and the center point of the lens of the telephoto camera 2 is respectively perpendicular to the radius of the arc groove 324 and the central axis of the lens of the telephoto camera 2. That is to say, the second mounting platform 34 can perform a conical pendulum motion relative to the first mounting platform 32, thereby prompting the telephoto camera 2 to perform a conical pendulum motion relative to the first mounting platform 32.

[0054] Optionally, the second motion assembly further includes a telescopic member 36 capable of achieving telescoping, one end of the telescopic member 36 is drivingly connected to the second driving member 35, and the other end of the telescopic member 36 is connected to the second carrying platform 34;

[0055] When the second driving member 35 drives the second carrying platform 34 through the telescopic member 36 , the telescopic member 36 is adapted to extend or shorten so that the guide block slides in the arc groove 324 .

[0056] Specifically, the telescopic member 36 includes a sleeve 361 and a moving rod 362, one end of the moving rod 362 is slidably connected to the sleeve 361, and the other end of the moving rod 362 is connected to the second mounting platform 34. The sleeve 361 is threadedly connected to the shaft of the second motor. When the second motor is working, the second motor drives the sleeve 361 to move along the axial direction of the shaft (the direction where the X-axis is located). Since the guide block is located in the arc groove 324, the second mounting platform 34 cannot make a linear motion along the direction where the X-axis is located. The moving rod 362 will extend or shorten relative to the sleeve 361 to make the guide block slide in the arc groove 324, so that the second mounting platform 34 drives the telephoto camera 2 to make a conical pendulum motion relative to the first mounting platform 32.

[0057] In this embodiment, the first motor is located above the second loading platform 34. In order to facilitate the connection of the moving rod to the second loading platform 34, a connecting column 341 is provided on the second loading platform 34. A latch is provided at the top of the connecting column 341. A pin hole matching the latch is provided on the moving rod. The latch is inserted into the pin hole to achieve connection. At the same time, in order to further strengthen the firmness of the connecting column 341 and the moving rod, the pin hole is a through hole, the latch is inserted into the pin hole, and the part of the latch extending out of the pin hole is connected to the moving rod by a snap.

[0058] Optionally, the eye tracking imaging system also includes a second limiting member 37, the second driving member 35 includes a second motor, the rotating shaft of the second motor is threadedly connected to the telescopic member 36, and the second limiting member 37 is located on the moving path of the telescopic member 36 to limit the second limiting member 37 from being separated from the rotating shaft of the second motor.

[0059] Specifically, the second stopper 37 is located near the free end of the shaft of the second motor. When the sleeve 361 of the telescopic member 36 moves to the free end of the shaft of the second motor, it is blocked by the second stopper 37, so the sleeve 361 of the telescopic member 36 will not be separated from the shaft of the second motor. Figure 2 As shown, the second limit member 37 is Z-shaped, one end of the second limit member 37 is fixed on the mounting plate 322 of the second mounting platform 34, and the other end of the second limit member 37 is located on the right side of the rotating shaft of the second motor. There is a gap between the part of the second limit member 37 located on the right side of the rotating shaft of the second motor and the rotating shaft of the second motor, and the distance of the gap is less than the depth of the threaded hole of the sliding sleeve, so the second limit member 37 will not hinder the movement of the rotating shaft of the second motor.

[0060] With such arrangement, under the action of the second limiting member 37 , the travel of the sliding sleeve can be limited to prevent the sliding sleeve from being separated from the rotating shaft of the second motor.

[0061] Optionally, the eye tracking imaging system further includes a fixing seat 4, a mounting frame 7 and a wide-angle camera 1, wherein the wide-angle camera 1 is used to obtain the facial image of each person in a dynamic and large-field-of-view multi-information environment in real time, a mounting groove 42 is provided in the fixing seat 4, the mounting frame 7 and the motion platform 3 are both arranged in the mounting groove 42, and the wide-angle camera 1 is installed on the mounting frame 7; an imaging port 41 communicating with the mounting groove 42 is provided on the side wall of the fixing seat 4, and the optical path difference between the wide-angle camera 1 and the telephoto camera 2 to the imaging port 41 is zero.

[0062] In this embodiment, the shape of the fixing seat 4 is not limited, and it can be square or round. Figure 1 , 4As shown, the fixing seat 4 is in the shape of a rectangle, and a mounting groove 42 is provided in the fixing seat 4, and the wide-angle camera 1, the telephoto camera 2 and the motion platform 3 are all arranged in the mounting groove 42. Specifically, the first driving member 31 of the motion platform 3 is fixed to the bottom of the mounting groove 42 by screws, and the first carrying platform 32 of the motion platform 3 is rotatably connected to the side wall of the mounting groove 42. Specifically, the first carrying platform 32 is provided with at least one connecting edge 323, and the connecting edge 323 is rotatably connected to the side wall of the mounting groove 42 by a rotating shaft, and the mounting frame 7 is connected to the side wall of the mounting groove 42 by screws. At the same time, the wide-angle camera 1 and the telephoto camera 2 are arranged side by side in the mounting groove 42 along the width direction of the fixing seat 4 (the direction where the X axis is located), and the lens of the wide-angle camera 1 and the lens of the telephoto camera 2 are in the same horizontal plane. The side wall of the fixing seat 4 is provided with an imaging port 41 that communicates with the mounting groove 42, and the optical path difference from the wide-angle camera 1 and the telephoto camera 2 to the imaging port 41 is zero.

[0063] In this embodiment, the eye tracking imaging further includes a lens assembly 8, and the imaging port 41 transmits light to the telephoto camera 2 and the wide-angle camera 1 respectively through the lens assembly 8. Specifically, the lens assembly 8 includes a semi-reflective half-mirror 81, which is tiltedly arranged at the imaging port 41, and the reflector 82 transmits the light reflected by the semi-reflective half-mirror 81 to the telephoto camera 2, and the reflector 82 transmits the light transmitted by the semi-reflective half-mirror 81 to the wide-angle camera 1. Figure 5 As shown, the lens assembly 8 further includes a plurality of reflectors 82. The light reflected by the semi-reflective and semi-mirror 81 is transmitted to the telephoto camera 2 through at least one reflector 82, and the light transmitted by the semi-reflective and semi-mirror 81 is transmitted to the wide-angle camera 1 through at least one reflector 82. Therefore, under the action of the lens assembly 8, the positions of the wide-angle camera 1 and the telephoto camera 2 can be reasonably arranged, thereby reducing the volume of the entire device.

[0064] In this embodiment, the semi-reflective mirror 81 and the reflector 82 are fixed in the mounting groove 42 by snap fastening, or are pasted in the mounting groove 42.

[0065] Optionally, the eye tracking imaging system further includes a control component 5 and a cover 6, wherein the control component 5 is disposed on the fixing base 4, and the control component 5 is electrically connected to the wide-angle camera 1, the telephoto camera 2 and the motion platform 3; the cover 6 is disposed on the fixing base 4, and the control component 5 is located inside the cover 6.

[0066] In this embodiment, Figure 1 As shown, the control component 5 includes a Raspberry Pi and a switch 52, and the Raspberry Pi is electrically connected to the wide-angle camera 1, the telephoto camera 2, the motion platform 3 and the switch 52 respectively. Figure 1 , 4 As shown, the cover 6 is disposed on the fixing base 4 , and the control component 5 is located inside the cover 6 , thereby achieving the purpose of protecting the control component 5 .

[0067] When working, the eye image information is reflected to the reflector 82 through the semi-reflective and semi-transmissive mirror, and then the reflector 82 is reflected to the telephoto camera 2, and the acquired eye image information is transmitted to the telephoto camera 2 in real time. At the same time, the eye image information is transmitted to the reflector 82 in real time through the semi-reflective and semi-transmissive mirror, and the reflector 82 transmits the eye image information to the wide-angle camera 1 in real time through the reflector 82. Since the optical path of the eye image information transmitted to the telephoto camera 2 is the same as that of the wide-angle camera 1, it is transmitted synchronously and in real time. The Raspberry Pi combines the wide-angle camera 1 and the telephoto camera 2 to build a local area network through the switch 52. Through image analysis, the coordinate difference between the target image position and the origin coordinate of the telephoto camera 2 is obtained, and the coordinate conversion is performed. The Raspberry Pi obtains the required pulse, sends the command pulse to the motion platform 3, controls the telephoto camera 2 to swing to the target position, and finally the telephoto camera 2 obtains a high-pixel eye image through automatic focus.

[0068] When shooting a target image to acquire eye information, the present invention adopts a combination of a wide-angle camera 1 and a telephoto camera 2, utilizing the characteristic of the wide-angle camera 1 that it can accommodate a wide range of objects and images to ensure that the target is always within the capture range, and the characteristic of the telephoto camera 2 that it can acquire images with high pixel counts, and then obtains the roll angle and pitch angle of the eyeball to be monitored and the optical axis of the telephoto camera 2, and finally adjusts the position of the telephoto camera 2 to align with the target eyeball image.

[0069] like Figure 6 As shown, an eye tracking imaging method according to another embodiment of the present invention adopts the eye tracking imaging system as described above, comprising:

[0070] S1. Obtain facial images of each person in real time in dynamic and large-field-of-view multi-information environments;

[0071] S2. Obtaining a roll angle and a pitch angle between the eyeball to be monitored and the optical axis of the telephoto camera 2 of the eye tracking imaging system according to the facial image;

[0072] S3, based on a preset conversion rule, converting the roll angle and the pitch angle into a motion instruction;

[0073] S4. Based on the motion instruction, control the motion platform 3 of the eye tracking imaging system to move.

[0074] In step S1, the facial image of each person in a dynamic and large-field-of-view multi-information environment is acquired by the wide-angle camera 1 of the eye tracking imaging system.

[0075] In step S2, according to face matching and an improved deep convolutional network (DCNN) model, facial feature information is extracted from the facial image to obtain the first real-time coordinates of the target image; according to the servo feedback of the first driver 31 and the second driver 35 of the eye tracking imaging system, the second real-time coordinates of the optical axis of the telephoto camera 2 in the facial image are obtained; according to the first real-time coordinates and the second real-time coordinates, the roll angle and the pitch angle of the eyeball to be monitored and the optical axis of the telephoto camera 2 are calculated. In step S3, according to the mechanical principle and the corresponding relationship between the swing angle of the telephoto camera 2, the roll angle and the pitch angle are converted into the transfer function of the roll angle and the pitch angle control of the telephoto camera 2, and the motion instruction for controlling the motion platform 3 is obtained through the transfer function. Since the first driver 31 and the second driver 35 of the motion platform 3 are both motors, the motion instruction includes the number of pulses.

[0076] In step S4 , after receiving the motion instruction (pulse number), the first motor and the second motor of the motion platform 3 drive the first mounting platform 32 and the second mounting platform 34 to move so as to enable the telephoto camera 2 to aim at the target.

[0077] Alternatively, if Figure 7 As shown, the method of controlling the motion platform 3 of the eye tracking imaging system to move based on the motion instruction also includes:

[0078] S5. When the motion platform 3 moves to the right position, the telephoto camera 2 is controlled to focus, and an image of micro features of the eye is acquired in real time through the telephoto camera 2.

[0079] In step S5, the telephoto camera 2 focuses after adjusting the roll angle and pitch angle of the telephoto camera 2. After the telephoto camera 2 acquires the eye micro-feature image, the eye micro-feature image is segmented by iris region, and the feature region of interest in the time series image is extracted; by establishing a mathematical model between the image feature and the human behavior pattern, the state of the target person is analyzed and predicted, and finally the obtained state of the target person is output.

[0080] In the description of this specification, the description with reference to the terms "embodiment", "one embodiment", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or implementation are included in at least one embodiment or implementation of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or implementation. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or implementations in a suitable manner.

[0081] Although the disclosure is disclosed as above, the protection scope of the disclosure is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the disclosure, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. An eye tracking imaging system, characterized in that: include: A telephoto camera (2) for acquiring images of eye micro-features in real time; A motion platform (3), comprising a first motion component, a second motion component, a first mounting platform (32) and a second mounting platform (34); the first motion component is drivingly connected to the telephoto camera (2) and is used to keep the focal length of the telephoto camera (2) fixed and drive the telephoto camera (2) to rotate about a first direction, so as to adjust the pitch angle of the lens of the telephoto camera (2); the second motion component is drivingly connected to the telephoto camera (2) and is used to keep the focal length of the telephoto camera (2) fixed and drive the telephoto camera (2) to rotate about a second direction, so as to adjust the roll angle of the lens of the telephoto camera (2); the telephoto camera (2) is arranged on the first mounting platform (32); the first motion component comprises a first driving component A moving member (31), the first driving member (31) is drivingly connected to the first mounting platform (32) and is used to drive the first mounting platform (32) to rotate about the first direction; the second moving assembly is arranged on the second mounting platform (34); the second mounting platform (34) is movably arranged on the first mounting platform (32); the telephoto camera (2) is arranged on the first mounting platform (32) via the second mounting platform (34); the second moving assembly comprises a second driving member (35), the second driving member (35) is drivingly connected to the second mounting platform (34) and is used to drive the second mounting platform (34) to rotate about the second direction relative to the first mounting platform (32); A wide-angle camera (1), the wide-angle camera (1) being used to obtain in real time a facial image of each person in a dynamic and large-field-of-view multi-information environment; A fixing seat (4), wherein a mounting groove (42) is provided in the fixing seat (4), a side wall of the fixing seat (4) is provided with an imaging port (41) communicating with the mounting groove (42), and an optical path difference between the wide-angle camera (1) and the telephoto camera (2) to the imaging port (41) is zero; A mounting frame (7), wherein the mounting frame (7) and the motion platform (3) are both arranged in the mounting groove (42), and the wide-angle camera (1) is mounted on the mounting frame (7).

2. The eye tracking imaging system according to claim 1, characterized in that: An arc groove (324) is provided on the first mounting platform (32), and a guide block is provided on the end surface of the second mounting platform (34) facing the first mounting platform (32), the guide block is slidably arranged in the arc groove (324), and a line connecting the center of the arc groove (324) and the center point of the lens of the telephoto camera (2) is respectively perpendicular to the radius of the arc groove (324) and the center axis of the lens of the telephoto camera (2).

3. The eye tracking imaging system according to claim 2, characterized in that: The second motion assembly further comprises a telescopic member (36) capable of achieving telescoping, one end of the telescopic member (36) being drivingly connected to the second driving member (35), and the other end of the telescopic member (36) being connected to the second carrying platform (34); When the second driving member (35) drives the second carrying platform (34) to move via the telescopic member (36), the telescopic member (36) is adapted to extend or shorten so that the guide block slides in the arc-shaped groove (324).

4. The eye tracking imaging system according to claim 1, characterized in that: The device further comprises a control component (5) and a cover (6), wherein the control component (5) is arranged on the fixing seat (4), and the control component (5) is electrically connected to the wide-angle camera (1), the telephoto camera (2) and the motion platform (3); the cover (6) is covered on the fixing seat (4), and the control component (5) is located inside the cover (6).

5. An eye tracking imaging method, using the eye tracking imaging system according to any one of claims 1 to 4, comprising: Real-time acquisition of facial images of each person in dynamic and large-field-of-view multi-information environments; According to the facial image, a roll angle and a pitch angle between the eyeball to be monitored and the optical axis of a telephoto camera (2) of the eye tracking imaging system are obtained; Based on a preset conversion rule, convert the roll angle and the pitch angle into a motion instruction; Based on the motion instruction, the motion platform (3) of the eye tracking imaging system is controlled to move.

6. The eye tracking imaging method according to claim 5, characterized in that: The step of obtaining the roll angle and pitch angle of the eyeball to be monitored and the optical axis of the telephoto camera (2) of the eye tracking imaging system according to the facial image comprises: Extracting facial feature information from the facial image based on face matching and an improved deep convolutional network model to obtain a first real-time coordinate of a target image; Obtaining a second real-time coordinate of the optical axis of the telephoto camera (2) in the facial image according to servo feedback of a first driving member (31) and a second driving member (35) of the eye tracking imaging system; The roll angle and the pitch angle between the eyeball to be monitored and the optical axis of the telephoto camera (2) are calculated according to the first real-time coordinate and the second real-time coordinate.

7. The eye tracking imaging method according to claim 5, characterized in that: The converting the roll angle and the pitch angle into a motion instruction based on a preset conversion rule comprises: According to mechanical principles and the corresponding relationship between the swing angle of the telephoto camera (2), the roll angle and the pitch angle are converted into a transfer function for controlling the roll angle and the pitch angle of the telephoto camera (2), and a motion instruction for controlling the motion platform (3) is obtained through the transfer function.