Calibration method and device of eye movement tracking camera equipment, equipment and storage medium

By generating calibration voice guidance information and using electrochromic strategies to display calibration key points, combined with eye tracking calibration algorithm, the problem of inability to calibrate on smart photography glasses is solved, and the calibration of eye tracking camera devices on non-display devices is realized , improves the user experience.

CN119094745AActive Publication Date: 2024-12-06QINGDAO GOERTEK VISION TECH CO LTD
View PDF 16 Cites 0 Cited by

Patent Information

Application Number
CN202411514301.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-29
Publication Date
2024-12-06
Estimated Expiration
2044-10-29

AI Technical Summary

Technical Problem

The prior art cannot realize calibration of eye tracking camera devices on smart photography glasses without display functions.

Method used

By generating calibration voice guidance information, the calibration key points are displayed on the lens using an electrochromic strategy, and the calibration camera device is calibrated according to the user's eye image through an eye tracking calibration algorithm.

Benefits of technology

The calibration of eye tracking camera devices on non-display devices is achieved, improving the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094745A_ABST
    Figure CN119094745A_ABST
Patent Text Reader

Abstract

The invention discloses a calibration method and device of eye movement tracking camera equipment, equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: generating calibration voice guide information when it is detected that a user correctly wears target intelligent equipment; when it is determined that the user closes the eyes opposite to the to-be-calibrated camera equipment according to the calibration voice guide information, displaying a calibration key point through an electrochromic strategy; when it is detected that the user watches the calibration key point, calibrating the to-be-calibrated camera device according to the eyeball picture and the displayed calibration key point through an eye movement tracking calibration algorithm; through the above mode, when it is determined that the user closes the eyes opposite to the to-be-calibrated camera equipment, the plurality of calibration key points are displayed on the lens by using the electrochromic strategy, and when the user watches the calibration key points, the to-be-calibrated camera equipment is calibrated by running the eye movement tracking calibration algorithm, so that the calibration accuracy of the to-be-calibrated camera equipment is improved. Therefore, the calibration of the eye movement tracking camera device on the non-display device can be realized, and the user experience is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a calibration method, apparatus, device and storage medium for an eye-tracking camera. Background Art

[0002] In recent years, smart camera glasses have attracted widespread attention from users due to their small size and light weight. The main functions of the smart camera glasses are taking pictures, listening to music, and making calls, and they do not have a display function. However, in order to increase interaction with users, the smart device needs to be equipped with an eye-tracking camera, such as an eye-tracking camera. However, the eye-tracking camera needs to be calibrated to accurately locate the visual direction of the human eye. At present, the common method of calibrating the eye-tracking camera is for smart devices with display functions, such as VR glasses. Specifically, the calibration is performed by the user staring at the points displayed on the screen. Calibration cannot be achieved for smart camera glasses without a display function. Therefore, the above method cannot calibrate the eye-tracking camera on a non-display device.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present application is to provide a calibration method, apparatus, device and storage medium for an eye-tracking camera device, aiming to solve the technical problem that the prior art cannot calibrate an eye-tracking camera device on a non-display device.

[0005] To achieve the above objectives, the present application proposes a calibration method for an eye tracking camera device, the method comprising: When it is detected that the user is wearing the target smart device correctly, a calibration voice guidance message is generated; When it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information, displaying calibration key points on the lens corresponding to the eye through an electrochromic strategy; When it is detected that the user is gazing at the calibration key point, obtaining an eyeball image of the user; The camera device to be calibrated is calibrated according to the eyeball image and the calibration key points through an eye tracking calibration algorithm.

[0006] In one embodiment, when it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information, the step of displaying calibration key points on the lens corresponding to the eye through an electrochromic strategy includes: Playing the calibration voice guidance information; Detecting in real time the state of the user's eyes corresponding to the camera device to be calibrated; When the state is a preset state, determining display parameters of the calibration key points; Calibration key points are displayed on the lens corresponding to the eye according to the display parameters through an electrochromic strategy.

[0007] In one embodiment, when the state is a preset state, the step of determining display parameters of the calibration key points includes: When the state is the preset state, obtaining historical sight data of the user after correctly wearing the target smart device; Obtaining attribute parameters of the target smart device; Determining a display position of the calibration key point according to the attribute parameters and the historical sight line data; Determining the display order of the calibration key points according to the calibration rules; The display parameters of the calibration key points are determined according to the display position, display order and display form.

[0008] In one embodiment, the step of calibrating the camera device to be calibrated according to the eye image and the calibration key points by using an eye tracking calibration algorithm includes: Acquire the user's eye pupil point according to the eye image; Determine the coordinates of the calibration key points in the camera coordinate system of the camera device to be calibrated; Determine the coordinates of the pupil point of the eyeball in the camera coordinate system of the camera device to be calibrated; The camera device to be calibrated is calibrated according to the coordinates of the calibration key points in the camera coordinate system of the camera device to be calibrated and the coordinates of the eye pupil points in the camera coordinate system of the camera device to be calibrated.

[0009] In one embodiment, the step of calibrating the camera device to be calibrated according to the coordinates of the calibration key points in the camera coordinate system of the camera device to be calibrated and the coordinates of the eye pupil point in the camera coordinate system of the camera device to be calibrated comprises: Binding the coordinates of the calibration key points in the camera coordinate system of the camera device to be calibrated with the coordinates of the eye pupil point in the camera coordinate system of the camera device to be calibrated; A fitting calculation is performed on the bound coordinate set, and calibration of the camera device to be calibrated is completed according to the fitting calculation result.

[0010] In one embodiment, after the step of calibrating the camera device to be calibrated according to the eye image and the calibration key points using an eye tracking calibration algorithm, the method further includes: When the temple area where the camera device to be calibrated is located is the left temple area, generating left-side calibration voice guidance information; When it is determined that the user closes the eye corresponding to the camera device to be calibrated on the right side according to the calibration voice guidance information on the left side, continue to perform the step of displaying calibration key points on the lens corresponding to the eye through the electrochromic strategy; When the temple area where the camera device to be calibrated is located is the right temple area, generating right-side calibration voice guidance information; When it is determined that the user closes the eye corresponding to the left camera device to be calibrated according to the right calibration voice guidance information, the step of displaying calibration key points on the lens corresponding to the eye through the electrochromic strategy is continued.

[0011] In one embodiment, after the step of calibrating the camera device to be calibrated according to the eye image and the calibration key points using an eye tracking calibration algorithm, the method further includes: The calibration key points are controlled to disappear from the lens corresponding to the eye through an electrochromic strategy.

[0012] In addition, to achieve the above-mentioned purpose, the present application also proposes a calibration device for an eye-tracking camera device, the calibration device for an eye-tracking camera device comprising: A generation module, used to generate calibration voice guidance information when it is detected that the user is correctly wearing the target smart device; A determination module, configured to display calibration key points on a lens corresponding to the eye through an electrochromic strategy when it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information; An acquisition module, configured to acquire a user's eyeball image when detecting that the user is gazing at the calibration key point; A calibration module is used to calibrate the camera device to be calibrated according to the eye image and the calibration key points through an eye tracking calibration algorithm.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes a calibration device for an eye-tracking camera device, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the calibration method for the eye-tracking camera device as described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the calibration method of the eye tracking camera device as described above are implemented.

[0015] One or more technical solutions proposed in the present application have at least the following technical effects: by generating calibration voice guidance information when it is detected that the user is correctly wearing the target smart device; when it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information, the calibration key points are displayed on the lens corresponding to the eye through the electrochromic strategy; when it is detected that the user is looking at the calibration key points, the user's eye picture is obtained; the camera device to be calibrated is calibrated according to the eye picture and the calibration key points through the eye tracking calibration algorithm; through the above method, when it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information, a plurality of calibration key points are displayed on the lens using the electrochromic strategy, and when the user looks at the calibration key points, the camera device to be calibrated is calibrated by running the eye tracking calibration algorithm, so that the eye tracking camera device on the non-display device can be calibrated, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the description, are used to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 A flowchart diagram of Embodiment 1 of the method for calibrating an eye tracking camera device of the present application; Figure 2 A schematic diagram of the structure of a target smart device for the eye tracking camera device calibration method provided in Embodiment 1 of the present application; Figure 3 A flowchart diagram of Embodiment 2 of the calibration method for an eye tracking camera device of the present application; Figure 4 A schematic diagram showing key calibration points of the eye tracking camera device calibration method provided in the second embodiment of the present application; Figure 5 This is a schematic diagram of the module structure of a calibration device for an eye tracking camera device according to an embodiment of the present application; Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the calibration method of the eye tracking camera device in the embodiment of the present application.

[0019] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a calibration device, etc. The following takes the calibration device as an example to illustrate this embodiment and the following embodiments.

[0021] Based on this, the present application embodiment provides a calibration method for an eye tracking camera device, referring to Figure 1 , Figure 1 This is a flowchart of a first embodiment of a calibration method for an eye tracking camera device of the present application.

[0022] In this embodiment, the calibration method of the eye tracking camera device includes steps S10 to S40: Step S10, when it is detected that the user is wearing the target smart device correctly, calibration voice guidance information is generated.

[0023] It should be noted that the target smart device refers to a smart device with functions such as taking pictures, listening to music, and making calls but without a display function. The target smart device can be a smart camera glasses, and the camera device to be calibrated can be an eye tracking camera. Figure 2 , Figure 2 : is a structural diagram of the target smart device, specifically: the target smart device includes three parts: left temple, right temple, and front frame, and the above parts can be electrically connected through FPC. The left temple and the right temple respectively include key components such as a main board and a battery. The target smart device is also provided with core components, including but not limited to an eye tracking module (which may also be referred to as a camera device to be calibrated before calibration), an electrochromic module, and a main control module. Among them, the eye tracking module can be located in a designated area of ​​the temple, which can be the inner side of the shaft, for taking pictures of the user's eyeballs. The electrochromic module is located on the lens, for converting the transmittance to display the calibration key points on the lens. The main control module is located on the temple, for system control, the operation of the eye tracking calibration algorithm, etc.

[0024] It should be understood that the calibration voice guidance information refers to voice information that guides and prompts the user to close their eyes and look at the calibration key points. For example, when the temple area where the camera to be calibrated is located is the left temple area, voice guidance information is generated to prompt the user to close the right eye and look at the calibration key points on the left lens, that is, the right-side calibration voice guidance information.

[0025] Step S20: When it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information, calibration key points are displayed on the lens corresponding to the eye through an electrochromic strategy.

[0026] It can be understood that the electrochromic strategy refers to a color-changing strategy that uses voltage to control the light transmittance of the target smart device. This electrochromic strategy is equivalent to adding a "simple display" function to the target smart device, that is, the calibration key points can be displayed on the lens of the target smart device.

[0027] Step S30, when it is detected that the user is gazing at the calibration key point, obtaining an eye image of the user.

[0028] It should be understood that when it is detected that the user is gazing at the calibration key points displayed on the lens, the camera device to be calibrated is controlled to take a picture of the user's eyeballs. It should be noted that there are multiple calibration key points. Therefore, each time the user gazes at a calibration key point, the camera device to be calibrated will be controlled to take a picture until the user has gazed at all the calibration key points.

[0029] Step S40: calibrating the camera device to be calibrated according to the eye image and the calibration key points by using an eye tracking calibration algorithm.

[0030] It can be understood that the eye tracking calibration algorithm refers to the algorithm used to calibrate the eye tracking module. After obtaining the eye image, it will be combined with the calibration key points and calibrated by running the eye tracking calibration algorithm. After the calibration is completed, the target smart device can accurately locate the visual direction of the human eye, increase the interaction between the target smart device and the user, and achieve the purpose of significantly improving the user experience.

[0031] Further, step S40 includes: obtaining the user's eye pupil point based on the eye image; determining the coordinates of the calibration key points in the camera coordinate system of the camera to be calibrated; determining the coordinates of the eye pupil point in the camera coordinate system of the camera to be calibrated; and calibrating the camera to be calibrated according to the coordinates of the calibration key points in the camera coordinate system of the camera to be calibrated and the coordinates of the eye pupil point in the camera coordinate system of the camera to be calibrated.

[0032] It should be understood that the eye pupil point is a characteristic point on the user's eyeball. The eye pupil is located in the center of the eye and can adjust the amount of light entering the eye. Since the relative positions of the lens and the camera to be calibrated are fixed, the relative positions of several calibration key points displayed on the lens in the camera coordinate system of the camera to be calibrated are also known, that is, the coordinates of the calibration key points in the camera coordinate system of the camera to be calibrated are determined. In addition, the user's eyeball can be photographed when the user is looking at the key points, so that the user's line of sight can be converted to the camera coordinate system. Therefore, the coordinates of the eye pupil point in the camera coordinate system of the camera to be calibrated can be determined, and then the camera to be calibrated is calibrated according to the coordinates of the calibration points in the camera coordinate system of the camera to be calibrated and the coordinates of the calibration key points in the camera coordinate system of the camera to be calibrated.

[0033] Furthermore, the step of calibrating the camera to be calibrated according to the coordinates of the calibration key points in the camera coordinate system of the camera to be calibrated and the coordinates of the eye pupil point in the camera coordinate system of the camera to be calibrated includes: binding the coordinates of the calibration key points in the camera coordinate system of the camera to be calibrated and the coordinates of the eye pupil point in the camera coordinate system of the camera to be calibrated; performing fitting calculation on the bound coordinate set, and completing the calibration of the camera to be calibrated according to the fitting calculation result.

[0034] It can be understood that after obtaining the coordinates of the calibration key points in the camera coordinate system of the camera device to be calibrated and the coordinates of the eye pupil point in the camera coordinate system of the camera device to be calibrated, the two are bound, and then the bound coordinate set can be fitted and calculated by a linear regression algorithm or a polynomial fitting algorithm to complete the calibration of the camera device to be calibrated. In addition, in order to further confirm whether the calibration meets the accuracy requirements, calibration verification can be performed.

[0035] Furthermore, after step S40, it also includes: when the temple area where the camera device to be calibrated is located is the left temple area, generating left-side calibration voice guidance information; when it is determined that the user closes the eye corresponding to the right camera device to be calibrated according to the left-side calibration voice guidance information, continuing to execute the step of displaying calibration key points on the lens corresponding to the eye through the electrochromic strategy; when the temple area where the camera device to be calibrated is located is the right temple area, generating right-side calibration voice guidance information; when it is determined that the user closes the eye corresponding to the left camera device to be calibrated according to the right-side calibration voice guidance information, continuing to execute the step of displaying calibration key points on the lens corresponding to the eye through the electrochromic strategy.

[0036] It should be understood that since the target smart device has multiple temples, for example, the left temple and the right temple, and a camera device to be calibrated is provided on the inner side of the hinge of each temple, multiple calibrations are required. After all the camera devices to be calibrated are calibrated, the calibration of the target smart device is completed.

[0037] It should be noted that when the temple area where the camera to be calibrated is located is the left temple area, it means that the first camera to be calibrated is the camera to be calibrated located on the inner side of the left temple axis. Then the next camera to be calibrated is the camera to be calibrated located on the inner side of the right temple axis, that is, the right camera to be calibrated. At this time, according to the generated left calibration voice guidance information, the user is prompted to close the eye corresponding to the right camera to be calibrated, that is, to close the left eye, and then continue to execute the step of calibrating the camera to be calibrated according to the eye picture and calibration key points through the eye tracking calibration algorithm. In addition, when the temple area where the camera to be calibrated is located is the right temple area, it indicates that the first thing to be calibrated is the camera to be calibrated located on the inner side of the right temple axis. Then the next thing to be calibrated is the camera to be calibrated located on the inner side of the left temple axis, that is, the left camera to be calibrated. Then, according to the generated right calibration voice guidance information, the user is prompted to close the eye corresponding to the left camera to be calibrated, that is, to close the right eye. Then, continue to execute the step of calibrating the camera to be calibrated according to the eye picture and calibration key points through the eye tracking calibration algorithm.

[0038] It should be understood that in the present embodiment, the playback device used to play the calibration voice guidance information can be one or more. When there is one playback device, both the left-side calibration voice guidance information and the right-side calibration voice guidance information are played by the playback device. When there are multiple playback devices, the calibration voice guidance information is played by the playback device on the same side. For example, if the camera device to be calibrated is located on the inner side of the right temple shaft, the playback device on the right side plays the left-side calibration voice guidance information, prompting the user to close the left eye. If the camera device to be calibrated is located on the inner side of the left temple shaft, the playback device on the left side plays the right-side calibration voice guidance information, prompting the user to close the right eye.

[0039] Furthermore, after the step of calibrating the camera device to be calibrated according to the eye image and the calibration key points through an eye tracking calibration algorithm, it also includes: controlling the calibration key points to disappear from the lens corresponding to the eye through an electrochromic strategy.

[0040] It is understandable that after the calibration is completed, the calibration key points are controlled to disappear from the lenses corresponding to the eyes through the electrochromic strategy, and the target smart device can now be used as ordinary glasses.

[0041] It should be noted that if the electrochromic strategy is a monochrome electrochromic strategy, then the electrochromic strategy can only be used to calibrate the camera device to be calibrated and can no longer be used as sunglasses; if the electrochromic strategy is a multicolor electrochromic strategy, then the calibration of the camera device to be calibrated needs to occupy one color.

[0042] This embodiment generates calibration voice guidance information when it is detected that the user is correctly wearing the target smart device; when it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information, the calibration key points are displayed on the lens corresponding to the eye through the electrochromic strategy; when it is detected that the user is looking at the calibration key points, the user's eye picture is obtained; the camera device to be calibrated is calibrated according to the eye picture and the calibration key points through the eye tracking calibration algorithm; through the above method, when it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information, a plurality of calibration key points are displayed on the lens by using the electrochromic strategy, and when the user looks at the calibration key points, the camera device to be calibrated is calibrated by running the eye tracking calibration algorithm, so that the eye tracking camera device on the non-display device can be calibrated, thereby improving the user experience.

[0043] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , step S20 includes steps S201 to S204: Step S201, playing the calibration voice guidance information.

[0044] It should be noted that the calibration voice guidance information refers to voice information that guides and prompts the user to close their eyes and look at the calibration key points. The user can be prompted to close their eyes and look at the calibration key points by playing the calibration voice guidance information. For example, if the calibration is of the left camera device to be calibrated, the user will be prompted to close the right eye, and the calibration key points will be displayed on the lens, prompting the user to look at the calibration key points.

[0045] Step S202: detecting in real time the state of the user's eyes that correspond to the camera device to be calibrated.

[0046] It can be understood that the state of the eyes includes but is not limited to the closed eyes state, the open eyes state, etc. During the process of playing the calibration voice guidance information, the state of the eyes corresponding to the camera device to be calibrated will be detected in real time. For example, if the left camera device to be calibrated is calibrated, the state of the user's right eye will be detected in real time.

[0047] Step S203, when the state is a preset state, determining display parameters of the calibration key points.

[0048] It should be understood that the preset state can be an open-eye state. After detecting the state of the glasses in real time, it is necessary to determine whether the state is the preset state. If so, it means that the user has closed the eyes corresponding to the camera device to be calibrated. At this time, the display parameters of the calibration key points are determined, and the display parameters include but are not limited to display position, display order, and display form.

[0049] Further, step S203 includes: when the state is a preset state, obtaining historical sight data of the user after correctly wearing the target smart device; obtaining attribute parameters of the target smart device; determining the display position of the calibration key points according to the attribute parameters and the historical sight data; determining the display order of the calibration key points according to the calibration rules; determining the display parameters of the calibration key points according to the display position, display order and display form.

[0050] It can be understood that the attribute parameters refer to the basic attribute parameters of the target smart device, which include but are not limited to the size of the lens of the target smart device, the curvature of the frame, and the center point of the lens, etc., and then the display position of the calibration key points is determined in combination with the historical line of sight data. The display order refers to the order in which multiple calibration key points are displayed through the electrochromic strategy.

[0051] It should be noted that the reference Figure 4 , Figure 4 This is a schematic diagram showing the calibration key points, specifically: taking 5 calibration key points as an example, namely the upper left calibration key point, the upper right calibration key point, the center calibration key point, the lower right calibration key point and the lower left calibration key point. At this time, the display order determined according to the calibration rule is the center calibration key point, the upper left calibration key point, the upper right calibration key point, the lower right calibration key point and the lower left calibration key point. The display form can be a solid circle that makes the user feel pleasant, and the display positions are the center of the lens, the upper left side, the upper right side, the lower left side and the lower right side.

[0052] Step S204: displaying calibration key points on the lens corresponding to the eye according to the display parameters through an electrochromic strategy.

[0053] It can be understood that the electrochromic strategy is equivalent to adding a "simple display" function to the target smart device. At this time, the calibration key points are displayed on the lens corresponding to the eye according to the display parameters, and the user can see the calibration key points.

[0054] This embodiment plays the calibration voice guidance information; detects the state of the user's eyes corresponding to the camera device to be calibrated in real time; when the state is a preset state, determines the display parameters of the calibration key points; and displays the calibration key points on the lenses corresponding to the eyes according to the display parameters through an electrochromic strategy; in the above manner, the user is prompted to close his eyes and look at the calibration key points by playing the calibration voice guidance information, and during the playback, the state of the user's eyes is detected in real time, and then it is determined whether the state is the preset state. If so, the calibration key points are displayed on the lenses through the "simple display" function added by the electrochromic strategy, so that the calibration key points can be displayed on the non-display device, thereby realizing the calibration of the eye tracking camera device on the non-display device.

[0055] This application also provides a calibration device for an eye tracking camera device, please refer to Figure 5 , the calibration device of the eye tracking camera device comprises: The generating module 10 is used to generate calibration voice guidance information when it is detected that the user correctly wears the target smart device.

[0056] The determination module 20 is used to display calibration key points on the lens corresponding to the eye through an electrochromic strategy when it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information.

[0057] The acquisition module 30 is used to acquire the user's eyeball image when it is detected that the user is gazing at the calibration key point.

[0058] The calibration module 40 is used to calibrate the camera device to be calibrated according to the eye image and the calibration key points by using an eye tracking calibration algorithm.

[0059] This embodiment generates calibration voice guidance information when it is detected that the user is correctly wearing the target smart device; when it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information, the calibration key points are displayed on the lens corresponding to the eye through the electrochromic strategy; when it is detected that the user is looking at the calibration key points, the user's eye picture is obtained; the camera device to be calibrated is calibrated according to the eye picture and the calibration key points through the eye tracking calibration algorithm; through the above method, when it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information, a plurality of calibration key points are displayed on the lens by using the electrochromic strategy, and when the user looks at the calibration key points, the camera device to be calibrated is calibrated by running the eye tracking calibration algorithm, so that the eye tracking camera device on the non-display device can be calibrated, thereby improving the user experience.

[0060] The calibration device for an eye-tracking camera device provided by the present application adopts the calibration method for an eye-tracking camera device in the above-mentioned embodiment, which can solve the technical problem that the prior art cannot calibrate an eye-tracking camera device on a non-display device. Compared with the prior art, the beneficial effects of the calibration device for an eye-tracking camera device provided by the present application are the same as the beneficial effects of the calibration method for an eye-tracking camera device provided by the above-mentioned embodiment, and other technical features in the calibration device for an eye-tracking camera device are the same as the features disclosed in the above-mentioned embodiment method, which will not be described in detail here.

[0061] In one embodiment, the determination module 20 is further used to play the calibration voice guidance information; detect in real time the state of the user's eyes corresponding to the camera device to be calibrated; when the state is a preset state, determine the display parameters of the calibration key points; and display the calibration key points on the lenses corresponding to the eyes according to the display parameters through an electrochromic strategy.

[0062] In one embodiment, the determination module 20 is also used to obtain historical line of sight data of the user after correctly wearing the target smart device when the state is a preset state; obtain attribute parameters of the target smart device; determine the display position of the calibration key points according to the attribute parameters and the historical line of sight data; determine the display order of the calibration key points according to the calibration rules; determine the display parameters of the calibration key points according to the display position, display order and display form.

[0063] In one embodiment, the calibration module 40 is further used to obtain the user's eye pupil point based on the eye image; determine the coordinates of the calibration key points in the camera coordinate system of the camera to be calibrated; determine the coordinates of the eye pupil point in the camera coordinate system of the camera to be calibrated; and calibrate the camera to be calibrated according to the coordinates of the calibration key points in the camera coordinate system of the camera to be calibrated and the coordinates of the eye pupil point in the camera coordinate system of the camera to be calibrated.

[0064] In one embodiment, the calibration module 40 is further used to bind the coordinates of the calibration key points in the camera coordinate system of the camera device to be calibrated with the coordinates of the eye pupil point in the camera coordinate system of the camera device to be calibrated; perform fitting calculation on the bound coordinate set, and complete the calibration of the camera device to be calibrated based on the fitting calculation result.

[0065] In one embodiment, the calibration module 40 is further used to generate left-side calibration voice guidance information when the temple area where the camera to be calibrated is located is the left temple area; when it is determined that the user closes the eye corresponding to the right camera to be calibrated according to the left-side calibration voice guidance information, continue to execute the step of displaying calibration key points on the lens corresponding to the eye through the electrochromic strategy; when the temple area where the camera to be calibrated is located is the right temple area, generate right-side calibration voice guidance information; when it is determined that the user closes the eye corresponding to the left camera to be calibrated according to the right-side calibration voice guidance information, continue to execute the step of displaying calibration key points on the lens corresponding to the eye through the electrochromic strategy.

[0066] In one embodiment, the calibration module 40 is further used to control the calibration key points to disappear from the lens corresponding to the eye through an electrochromic strategy.

[0067] The present application provides a calibration device for an eye-tracking camera device, and the calibration device for the eye-tracking camera device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the calibration method for the eye-tracking camera device in the above-mentioned embodiment one.

[0068] Reference below Figure 6 , which shows a schematic diagram of the structure of a calibration device for an eye-tracking camera device suitable for implementing an embodiment of the present application. The calibration device for the eye-tracking camera device in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The calibration device of the eye tracking camera device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0069] like Figure 6As shown, the calibration device of the eye tracking camera device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM) 1004. Various programs and data required for the operation of the calibration device of the eye tracking camera device are also stored in RAM 1004. The processing device 1001, ROM 1002, and RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 may allow the calibration device of the eye tracking camera device to communicate with other devices wirelessly or by wire to exchange data. Although the calibration device of the eye tracking camera device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0070] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. The computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0071] The calibration device for the eye-tracking camera device provided by the present application adopts the calibration method for the eye-tracking camera device in the above-mentioned embodiment, which can solve the technical problem that the prior art cannot calibrate the eye-tracking camera device on the non-display device. Compared with the prior art, the beneficial effects of the calibration device for the eye-tracking camera device provided by the present application are the same as the beneficial effects of the calibration method for the eye-tracking camera device provided by the above-mentioned embodiment, and the other technical features in the calibration device for the eye-tracking camera device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0072] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0073] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0074] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the calibration method of the eye-tracking camera device in the above-mentioned embodiment.

[0075] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0076] The computer-readable storage medium may be included in the calibration device of the eye-tracking camera device; or may exist independently without being assembled into the calibration device of the eye-tracking camera device.

[0077] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0078] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system and method according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0079] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0080] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the calibration method of the above-mentioned eye-tracking camera device, and can solve the technical problem that the prior art cannot calibrate the eye-tracking camera device on a non-display device. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the calibration method of the eye-tracking camera device provided by the above-mentioned embodiment, and will not be repeated here.

[0081] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for calibrating an eye tracking camera device, characterized in that: The method comprises: When it is detected that the user is wearing the target smart device correctly, a calibration voice guidance message is generated; When it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information, displaying calibration key points on the lens corresponding to the eye through an electrochromic strategy; When it is detected that the user is gazing at the calibration key point, obtaining an eyeball image of the user; The camera device to be calibrated is calibrated according to the eyeball image and the calibration key points through an eye tracking calibration algorithm.

2. The method according to claim 1, characterized in that The step of displaying calibration key points on the lens corresponding to the eye through an electrochromic strategy when it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information comprises: Playing the calibration voice guidance information; Detecting in real time the state of the user's eyes corresponding to the camera device to be calibrated; When the state is a preset state, determining display parameters of the calibration key points; Calibration key points are displayed on the lens corresponding to the eye according to the display parameters through an electrochromic strategy.

3. The method according to claim 2, characterized in that The step of determining the display parameters of the calibration key points when the state is the preset state includes: When the state is the preset state, obtaining historical sight data of the user after correctly wearing the target smart device; Obtaining attribute parameters of the target smart device; Determining a display position of the calibration key point according to the attribute parameters and the historical sight line data; Determining the display order of the calibration key points according to the calibration rules; The display parameters of the calibration key points are determined according to the display position, display order and display form.

4. The method according to claim 1, characterized in that The step of calibrating the camera device to be calibrated according to the eye image and the calibration key points by using an eye tracking calibration algorithm comprises: Acquire the user's eye pupil point according to the eye image; Determine the coordinates of the calibration key points in the camera coordinate system of the camera device to be calibrated; Determine the coordinates of the pupil point of the eyeball in the camera coordinate system of the camera device to be calibrated; The camera device to be calibrated is calibrated according to the coordinates of the calibration key points in the camera coordinate system of the camera device to be calibrated and the coordinates of the eye pupil points in the camera coordinate system of the camera device to be calibrated.

5. The method according to claim 4, characterized in that The step of calibrating the camera device to be calibrated according to the coordinates of the calibration key points in the camera coordinate system of the camera device to be calibrated and the coordinates of the eye pupil points in the camera coordinate system of the camera device to be calibrated comprises: Binding the coordinates of the calibration key points in the camera coordinate system of the camera device to be calibrated with the coordinates of the eye pupil point in the camera coordinate system of the camera device to be calibrated; A fitting calculation is performed on the bound coordinate set, and calibration of the camera device to be calibrated is completed according to the fitting calculation result.

6. The method according to claim 1, characterized in that After the step of calibrating the camera device to be calibrated according to the eyeball picture and the calibration key points by using an eye tracking calibration algorithm, the method further includes: When the temple area where the camera device to be calibrated is located is the left temple area, generating left-side calibration voice guidance information; When it is determined that the user closes the eye corresponding to the camera device to be calibrated on the right side according to the calibration voice guidance information on the left side, continue to perform the step of displaying calibration key points on the lens corresponding to the eye through the electrochromic strategy; When the temple area where the camera device to be calibrated is located is the right temple area, generating right-side calibration voice guidance information; When it is determined that the user closes the eye corresponding to the left camera device to be calibrated according to the right calibration voice guidance information, the step of displaying calibration key points on the lens corresponding to the eye through the electrochromic strategy is continued.

7. The method according to any one of claims 1 to 6, characterized in that After the step of calibrating the camera device to be calibrated according to the eyeball picture and the calibration key points by using an eye tracking calibration algorithm, the method further includes: The calibration key points are controlled to disappear from the lens corresponding to the eye through an electrochromic strategy.

8. A calibration device for an eye tracking camera, characterized in that: The device comprises: A generation module, used to generate calibration voice guidance information when it is detected that the user is correctly wearing the target smart device; A determination module, configured to display calibration key points on a lens corresponding to the eye through an electrochromic strategy when it is determined that the user closes the eye corresponding to the camera device to be calibrated according to the calibration voice guidance information; An acquisition module, configured to acquire a user's eyeball image when detecting that the user is gazing at the calibration key point; A calibration module is used to calibrate the camera device to be calibrated according to the eye image and the calibration key points through an eye tracking calibration algorithm.

9. A calibration device for an eye tracking camera device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the calibration method of the eye tracking camera device according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the calibration method of the eye-tracking camera device according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • An eyeball tracking method and device

    CN109558012A

  • Method of calibrating eye-tracking application

    CN111399633A

  • Eyeball tracking calibration method, device and system

    CN112149453A

  • Eyeball tracking calibration coefficient determination method and device, equipment and storage medium

    CN113534945A

  • Positioning visual field expanding method based on eye movement tracking principle and intelligent glasses

    CN115580692A