Gaze depth acquisition method and device, display device, and storage medium

Abstract

The present disclosure relates to a method and apparatus for acquiring gaze depth, a display device, and a storage medium. The method includes: acquiring an eye region of a target object in an input image; acquiring the center position of the pupil in the eye region and calculating the distance between the two pupils based on the center position; based on a correspondence between a preset pupil distance and a gaze depth, acquiring a first gaze depth of the target object based on the distance. In this embodiment, only an ordinary camera or a camera is needed to acquire the input image, without relying on professional equipment, which can reduce costs; and, in this embodiment, the above-mentioned input image is used to obtain the eye gaze depth in real time, thereby breaking through the limitation of the related technology that the eye gaze point is limited to a two-dimensional plane, and is suitable for the use scenario of the gaze depth in three-dimensional space.

Description

Technical Field

[0001] The present disclosure relates to the field of display technology, and in particular to a gaze depth acquisition method and apparatus, a display device, and a storage medium. Background Art

[0002] With the development of science and technology, especially the development of virtual reality technology, users can control electronic devices through eye movements. This control method is generally achieved by using eye tracking technology to track the user's gaze point and perform various control operations based on the tracking results.

[0003] The currently used eye tracking methods can only guarantee the accuracy of identifying the user's actual gaze point when the user is looking at objects at different positions in the two-dimensional plane direction, that is, they can only accurately describe the correspondence between the gaze point and the eye position in the plane direction. When the user is looking at two objects with the same two-dimensional position but different depths of field, the existing eye tracking technology can only determine the user's two-dimensional gaze point, but cannot accurately determine the user's actual gaze depth. The accuracy of gaze point identification and tracking cannot be guaranteed in three-dimensional application scenarios. Summary of the invention

[0004] The present disclosure provides a gaze depth acquisition method and apparatus, a display device, and a storage medium to address the deficiencies of related technologies.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for acquiring gaze depth is provided, comprising:

[0006] Get the eye region of the target object in the input image;

[0007] Acquire the center position of the pupil in the eye area and calculate the distance between the two pupils according to the center position;

[0008] Based on a preset correspondence between the pupil distance and the gaze depth, a first gaze depth of the target object is acquired according to the distance.

[0009] Optionally, obtaining the eye region of the target object in the input image includes:

[0010] Acquire a facial region of a target object in an input image; the facial region includes a first number of facial feature points;

[0011] A second number of eye feature points are extracted from the facial feature points, and an eye region of the target object is determined according to the second number of eye feature points.

[0012] Optionally, obtaining the center position of the pupil in the eye region and calculating the distance between two pupils according to the center position includes:

[0013] Acquire a left eye region and a right eye region in the eye region, and determine the center positions of the pupils in the left eye region and the right eye region respectively;

[0014] The distance between the two pupils is calculated based on the center position of the pupils in the left and right eyes.

[0015] Optionally, based on a preset correspondence between an interpupillary distance and a gaze depth, acquiring a first gaze depth of the target object according to the distance includes:

[0016] Obtaining a correspondence between a preset pupil distance and a gaze depth;

[0017] Based on a preset correspondence between the pupil distance and the gaze depth, a first gaze depth of the target object is acquired according to the distance.

[0018] Optionally, obtaining a correspondence between a preset pupil distance and a gaze depth includes:

[0019] When the relative positions of the eye and the camera are fixed, the distance between the eye and the movable device being gazed is obtained, and the distance is the gaze depth;

[0020] Obtain the distance between the two pupils when the mobile device being watched is stationary;

[0021] Polynomial fitting is performed on multiple groups of gaze depths and distances to obtain an objective function of gaze depth and pupil distance, wherein the objective function is used as a corresponding relationship between a preset pupil distance and gaze depth.

[0022] Optionally, the method further comprises:

[0023] The first gaze depth is optimized to obtain a third gaze depth, where the third gaze depth is the gaze depth after the first gaze depth is optimized.

[0024] Optionally, optimizing the first gaze depth to obtain a third gaze depth includes:

[0025] Scaling the eye area according to a preset scaling ratio to obtain a scaled eye area, so as to obtain a second gaze depth according to the scaled eye area;

[0026] A third gaze depth is acquired according to the first gaze depth and the second gaze depth.

[0027] Optionally, scaling the eye region according to a preset scaling ratio to obtain a scaled eye region includes:

[0028] When the relative positions of the camera and the movable device are fixed, the horizontal distance between the eye and the camera is obtained, and the length of the eye area is obtained;

[0029] Perform polynomial fitting on multiple groups of horizontal distances and lengths to obtain the objective function of the horizontal distance and length;

[0030] Obtain a first length of the eye region when the eye is at a first distance from the camera, and obtain a second length of the eye region when the eye is at a second distance from the camera, and calculate a ratio of the first length to the second length, which is used as a scaling ratio.

[0031] Optionally, acquiring a third gaze depth according to the first gaze depth and the second gaze depth includes:

[0032] Obtaining the horizontal distance between the eye and the camera at the second gaze depth;

[0033] Obtaining the sum of the second gaze depth and the horizontal distance to obtain an optimized gaze depth;

[0034] The first gaze depth and the optimized gaze depth are weighted averaged to obtain an average gaze depth, which is the third gaze depth.

[0035] According to a second aspect of an embodiment of the present disclosure, a gaze depth acquisition device is provided, comprising an eye region detection module, a pupil distance acquisition module and a gaze depth acquisition module; wherein,

[0036] The eye region acquisition module is used to acquire the eye region of the target object in the input image;

[0037] The pupil distance acquisition module is used to acquire the center position of the pupil in the eye area and calculate the distance between the two pupils according to the center position;

[0038] The gaze depth acquisition module is used to acquire a first gaze depth of the target object according to a preset correspondence between the pupil distance and the gaze depth.

[0039] Optionally, the eye region acquisition module includes a face region acquisition unit and an eye region acquisition unit;

[0040] The facial region acquisition unit is used to acquire the facial region of the target object in the input image; the facial region includes a first number of facial feature points;

[0041] The eye region acquisition unit is used to extract a second number of eye feature points from the facial feature points, and determine the eye region of the target object according to the second number of eye feature points.

[0042] Optionally, the pupil distance acquisition module includes a pupil positioning unit and a distance calculation unit;

[0043] The pupil positioning unit is used to obtain a left eye area and a right eye area in the eye area, and determine the center positions of the pupils in the left eye area and the right eye area respectively;

[0044] The distance calculation unit is used to calculate the distance between the two pupils according to the center positions of the pupils in the left eye and the right eye.

[0045] Optionally, the gaze depth acquisition module includes a correspondence acquisition unit and a gaze depth acquisition unit;

[0046] The corresponding relationship acquisition unit is used to acquire the corresponding relationship between the preset pupil distance and the gaze depth;

[0047] The gaze depth acquisition unit is configured to acquire a first gaze depth of the target object according to a preset pupil distance and a corresponding relationship between the pupil distance and the gaze depth.

[0048] Optionally, the correspondence relationship acquisition unit includes a gaze depth acquisition subunit, a pupil distance acquisition subunit and a correspondence relationship acquisition subunit;

[0049] The gaze depth acquisition subunit is used to acquire the distance between the eye and the gazed movable device when the relative positions of the eye and the camera are fixed, and the distance is the gaze depth;

[0050] The pupil distance acquisition subunit is used to acquire the distance between the two pupils when the movable device being watched is stationary;

[0051] The correspondence acquisition subunit is used to perform polynomial fitting on multiple groups of gaze depths and distances to obtain an objective function of gaze depth and pupil distance, wherein the objective function is a preset correspondence between pupil distance and gaze depth.

[0052] Optionally, a gaze depth optimization module is also included;

[0053] The gaze depth optimization module is used to optimize the first gaze depth to obtain a third gaze depth, where the third gaze depth is the gaze depth after the first gaze depth is optimized.

[0054] Optionally, the gaze depth optimization module includes an eye region scaling unit, a gaze depth acquisition unit and a gaze depth optimization unit;

[0055] The eye region scaling unit is used to scale the eye region according to a preset scaling ratio to obtain a scaled eye region and output the scaled eye region to the interpupillary distance acquisition module;

[0056] The gaze depth acquisition unit is used to acquire a second gaze depth output by the gaze depth acquisition module;

[0057] The gaze depth optimization unit is configured to acquire a third gaze depth according to the first gaze depth and the second gaze depth, where the third gaze depth is a gaze depth after the first gaze depth is optimized.

[0058] Optionally, the eye region scaling unit includes a distance length acquisition subunit, a distance length fitting subunit and a scaling ratio acquisition subunit;

[0059] The distance length acquisition subunit is used to acquire the horizontal distance between the eye and the camera, and the length of the eye area when the relative positions of the camera and the movable device are fixed;

[0060] The distance length fitting subunit is used to perform polynomial fitting on multiple groups of horizontal distances and lengths to obtain an objective function on the horizontal distances and lengths;

[0061] The scaling ratio acquisition subunit is used to obtain a first length corresponding to the eye area when the eye is at a first distance from the camera and to obtain a second length corresponding to the eye area when the eye is at a second distance from the camera, and to calculate a ratio of the first length to the second length, which is used as the scaling ratio.

[0062] Optionally, the gaze depth optimization unit includes:

[0063] A horizontal distance acquisition subunit, used for acquiring the horizontal distance between the eye and the camera at a second gaze depth;

[0064] An optimized depth acquisition subunit, used to acquire the sum of the second gaze depth and the horizontal distance to obtain an optimized gaze depth;

[0065] The third depth acquisition subunit is used to perform weighted average processing on the first gaze depth and the optimized gaze depth to acquire an average gaze depth, and the average gaze depth is the third gaze depth.

[0066] According to a third aspect of an embodiment of the present disclosure, there is provided a display device, including:

[0067] processor;

[0068] a memory for storing a computer program executable by the processor;

[0069] The processor is configured to execute the computer program in the memory to implement the above method.

[0070] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, and when an executable computer program in the storage medium is executed by a processor, the above method can be implemented.

[0071] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0072] As can be seen from the above embodiments, the eye region of the target object in the input image can be obtained in the disclosed embodiments; then, the center position of the pupil in the eye region is obtained and the distance between the two pupils is calculated according to the center position; then, based on the correspondence between the preset pupil distance and the gaze depth, the first gaze depth of the target object is obtained according to the distance. In this way, in this embodiment, only an ordinary camera or a camera is needed to obtain the input image, and the eye gaze depth is obtained in real time using the above input image, thereby breaking through the limitation of the eye gaze point in the related art that is limited to a two-dimensional plane, and being suitable for the use scenario of the gaze depth in three-dimensional space.

[0073] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0075] Figure 1 The figure is a flow chart of a method for acquiring gaze depth according to an exemplary embodiment.

[0076] Figure 2 The figure is a flow chart of obtaining an eye area according to an exemplary embodiment.

[0077] Figure 3 FIG. 4 is a schematic diagram showing the effect of a face area according to an exemplary embodiment.

[0078] Figure 4 It is a schematic diagram showing the effect of the eye area according to an exemplary embodiment.

[0079] Figure 5 The figure is a flow chart of obtaining pupil distance according to an exemplary embodiment.

[0080] Figure 6 It is a schematic diagram showing the relationship between the pupil distance and the gaze depth in the world coordinate system according to an exemplary embodiment.

[0081] Figure 7 The figure is a schematic diagram of an experimental scenario for obtaining the corresponding relationship between pupil distance and gaze depth according to an exemplary embodiment.

[0082] Figure 8 The flowchart of obtaining a first gaze depth is shown according to an exemplary embodiment.

[0083] Fig. 9 The figure is a flow chart of another method for acquiring gaze depth according to an exemplary embodiment.

[0084] Fig.10 is a flow chart of obtaining a third gaze depth according to an exemplary embodiment.

[0085] Fig.11 The figure is a schematic diagram of an experimental scenario for obtaining a scaling ratio according to an exemplary embodiment.

[0086] Fig.12 The figure is a flow chart of obtaining a zoom ratio according to an exemplary embodiment.

[0087] Fig.13 It is a schematic diagram showing the effect of the length change of the eye area before and after scaling according to an exemplary embodiment.

[0088] Fig.14 The figure is a block diagram of a device for acquiring gaze depth according to an exemplary embodiment. DETAILED DESCRIPTION

[0089] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described below by way of example do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0090] In order to solve the above technical problems, the embodiments of the present disclosure provide a method for acquiring gaze depth, which can be applied to display devices, such as mobile terminals such as smart phones and tablet computers, or fixed terminals such as desktop computers and televisions. Figure 1 FIG. 1 is a flow chart of a method for acquiring gaze depth according to an exemplary embodiment. Figure 1 , a gaze depth acquisition method, comprising steps 11 to 13:

[0091] In step 11, the eye region of the target object in the input image is obtained.

[0092] In this embodiment, the display device may be provided with an image acquisition device, such as an infrared camera, an infrared camera, a general camera or a general camera. It is understandable that, since the subsequent scheme involves determining the center position of the pupil, combined with the absorption of light by the iris, the image obtained by using an infrared camera or a camera has higher accuracy. The technicians may select the corresponding image acquisition device according to the specific scenario, and the corresponding scheme falls within the protection scope of the present disclosure.

[0093] After the target object (such as a person or an animal, etc.) is within the preset range, the image acquisition device can acquire an image within the preset range, that is, obtain an image containing the target object. When the image can be acquired, a suitable image acquisition device can be selected according to the specific scene, and the corresponding scheme falls within the protection scope of the present disclosure. Of course, the display device may not be provided with an image acquisition device. In this case, the display device communicates with an external image acquisition device, and the image is acquired through the external image acquisition device, which can also implement the scheme of the present disclosure.

[0094] In this embodiment, the image acquisition device can communicate with the display device, and after acquiring the image, it can output it to the processor of the display device, and the processor can use the acquired image as the input image. Of course, the image acquisition device can also store the acquired image in a designated storage area, so that the processor in the display device can read the image from the designated storage area as the input image.

[0095] In one embodiment, see Figure 2 In step 21, the processor in the display device may obtain the face area of ​​the target object in the input image, and the face area includes a first number of facial feature points. For example, a face detection model may be provided in the display device, and after obtaining the input image, the processor may input the input image into the face detection model, and the face detection model may include at least one of the following: a convolutional neural network, a recurrent neural network, etc. In this way, the display device may use the face detection model to determine the face area of ​​the target object in the input image. Figure 3 A face region including 68 face feature points is shown. In step 22, the processor in the display device may extract a second number of eye feature points from the face feature points, and determine the eye region of the target object according to the second number of eye feature points. Figure 4 The eye region containing 12 feature points is shown. Figure 4 , eye feature points 37 to 40 constitute the left eye area, and eye feature points 43 to 48 constitute the right eye area.

[0096] It should be noted that the image acquisition device may also be provided with a face detection model, and when a face is detected in an image, the image is output to the processor or stored in a designated storage area, and images that do not contain a face area are removed, thereby reducing the number of images processed by the processor and improving processing efficiency. Figure 2 The solution of the embodiment shown directly outputs the eye area in the image. In other words, the processor in the display device can obtain the eye area in the input image while obtaining the input image, which can further improve the processing efficiency. The technician can select a suitable solution for obtaining the eye area according to the specific scenario, and the corresponding solution falls within the protection scope of the present disclosure.

[0097] In step 12, the center position of the pupil in the eye region is obtained and the distance between the two pupils is calculated according to the center position.

[0098] In this embodiment, see Figure 5 In step 51, the processor in the display device can obtain the left eye area and the right eye area in the eye area, and determine the center position of the pupil in the left eye area and the right eye area respectively. For example, after determining the left eye area, the processor can cut out a left eye sub-image containing the left eye in the input image according to the left eye area, and then calculate the sum of the grayscale values ​​of each row and each column of the left eye sub-image respectively, and obtain a row grayscale value map according to the row coordinates and the sum of the grayscale values ​​of each row of pixels, and obtain a column grayscale value map according to the sum of the column coordinates and the grayscale values ​​of each column of pixels. The coordinate value of the lowest point in the row grayscale value map and the column grayscale value map is the center position of the pupil. It should be noted that the method for obtaining the center position of the pupil of the right eye can refer to the scheme of step 51, which will not be repeated here. In step 52, the processor in the display device can calculate the distance between the two pupils according to the center positions of the pupils in the left eye and the right eye. For example,

[0099] The center position of the left pupil is (PX1, PY1), and the center position of the right pupil is (PX2, PY2). The distance PCD is:

[0100]

[0101] In step 13, based on a preset correspondence between the pupil distance and the gaze depth, a first gaze depth of the target object is acquired according to the distance.

[0102] See also Figure 6According to the prior knowledge of the relationship between the pupil distance PCD' and the eye's gaze depth D in the world coordinate system: PCD' is positively correlated with D. In addition, the pupil distance PCD' in the world coordinate system and the pupil distance PCD in the camera coordinate system are mapped to each other. Considering that the pupil distance PCD' changes very little in the world coordinate system, and the coordinate conversion between the camera coordinate system and the world coordinate system is required, the amount of data processing will be relatively large. Therefore, in this embodiment, the corresponding relationship between the pupil distance PCD and the gaze depth D can be directly established. See Figure 7 , the experiment set up to obtain the corresponding relationship between the pupil distance PCD and the gaze depth D:

[0103] The position of the camera (i.e., image acquisition device) is fixed, and a movable device is placed behind the camera (away from the target object). There is a white circular pattern in the center of the device that is level with the eye. Keeping the relative position of the eye and the camera unchanged, move the device forward and backward to change the eye's gaze depth D, and measure the corresponding pupil distance PCD. Repeat the above process several times (e.g., 30 times) to obtain 30 sets of pupil distance and gaze depth. After removing abnormal data, perform polynomial fitting to obtain the objective function, as shown in the following formula:

[0104] D=a 10 +a 11 *PCD+a 12 *PCD 2 +a 13 *PCD 3 ,

[0105] In the formula, a 10 ,a 11 ,a 12 ,a 13 Represents the coefficient of the objective function.

[0106] In this embodiment, see Figure 8 In step 81, the processor in the display device may obtain a correspondence between a preset interpupillary distance and a gaze depth. In step 82, the processor may obtain a first gaze depth of the target object according to the distance based on the correspondence between the preset interpupillary distance and the gaze depth.

[0107] In actual application, in step 81, the display device may store the correspondence between the preset pupil distance and the gaze depth. At this time, the processor may directly read the correspondence from the storage area, so that the processor only needs to query the correspondence, which is conducive to improving the processing speed. Of course, in step 81, the processor may also directly obtain the correspondence between the preset pupil distance and the gaze depth. After obtaining the pupil distance, the processor may obtain the gaze depth according to the correspondence and the distance.

[0108] At this point, the disclosed embodiment can obtain the eye region of the target object in the input image; then, obtain the center position of the pupil in the eye region and calculate the distance between the two pupils according to the center position; then, based on the correspondence between the preset pupil distance and the gaze depth, obtain the first gaze depth of the target object according to the distance. In this way, in this embodiment, only an ordinary camera or a camera is needed to obtain the input image, without relying on professional equipment, which can reduce costs; and in this embodiment, the eye gaze depth is obtained in real time using the above input image, which can be applicable to the use scenario of gaze depth in three-dimensional space.

[0109] Fig. 9 The figure is a flow chart of another method for acquiring gaze depth according to an exemplary embodiment.

[0110] See also Fig. 9 , a gaze depth acquisition method, comprising steps 91 to 94:

[0111] In step 91, the eye region of the target object in the input image is obtained.

[0112] In this embodiment, the solution of step 91 is the same as the solution of step 11. For details, please refer to Figure 1 The contents of step 11 will not be repeated here.

[0113] In step 92, the center position of the pupil in the eye region is obtained and the distance between the two pupils is calculated according to the center position.

[0114] In this embodiment, the solution of step 92 is the same as the solution of step 12. For details, see Figure 1 The contents of step 13 will not be repeated here.

[0115] In step 93, based on a preset correspondence between the pupil distance and the gaze depth, a first gaze depth of the target object is acquired according to the distance.

[0116] In this embodiment, the solution of step 93 is the same as the solution of step 13. For details, please refer to Figure 1 The contents of step 13 will not be repeated here.

[0117] In step 94, the first gaze depth is optimized to obtain a third gaze depth, where the third gaze depth is the gaze depth after the first gaze depth is optimized.

[0118] In this embodiment, see Fig.10In step 101, the processor in the display device can scale the eye area according to a preset scaling ratio to obtain a scaled eye area, so as to obtain a second gaze depth according to the scaled eye area. In step 102, the processor in the display device can obtain a third gaze depth according to the first gaze depth and the second gaze depth. In this embodiment, the input image collected at a long distance is scaled by using the characteristics of the camera, which can simulate the gaze depth of the eye when the distance between the eye and the camera becomes closer, and the final gaze depth can be calculated by combining the distance and the nearness.

[0119] In step 101, the display device may store a preset zoom ratio, and the processor may read the preset zoom ratio. The processor may also calculate the preset zoom ratio in real time, see Figure 7 , Fig.11 and Fig.12 In step 121, the processor can obtain the horizontal distance d between the eye and the camera and the length l of the eye area when the relative positions of the camera and the movable device are fixed. The effect is as follows: Fig.13 As shown. The length may include at least one of the following: width or height. The processor may repeat the above process several times (e.g., 30 times), and after removing abnormal data, 30 sets of horizontal distances d and lengths l may be obtained. In step 122, the processor may perform polynomial fitting using the above multiple sets of horizontal distances and lengths l to obtain the objective function, as shown in the following formula:

[0120] d=a 20 +a 21 *l+a 22 *l 2 +a 23 *l 3 ;

[0121] In the formula, a 20 ,a 21 ,a 22 ,a 23 Represents the coefficient of the objective function.

[0122] In step 123, the processor may obtain a first length of the eye region when the eye is at a first distance from the camera and a second length of the eye region when the eye is at a second distance from the camera, and calculate a ratio of the first length to the second length, and use the ratio as a scaling ratio. For example, if the length of the eye region in the image captured by the camera is l1 when the eye is at a horizontal distance d1 from the camera, and the length of the eye region in the image captured by the camera is l2 when the eye is at a horizontal distance d2 from the camera, then the scaling ratio is

[0123] In step 102, the processor may obtain a second gaze depth After that, the second gaze depth is the change in the gaze depth before and after the zoom. To this end, the processor can obtain the horizontal distance dd between the eye and the camera, and then use the second gaze depth Combined with the horizontal distance dd to obtain the optimal gaze depth Optimizing the depth of gaze That is, the gaze depth from the eye to the zoomed image. Afterwards, the processor may perform weighted average processing according to the first gaze depth and the optimized gaze depth to obtain an average gaze depth, which is the third gaze depth after the first gaze depth is optimized.

[0124] At this point, in the embodiment of the present disclosure, the first gaze depth can be optimized to obtain the third gaze depth, which can improve the accuracy of the gaze depth and avoid the problem that the accuracy of the first gaze depth is affected by the smaller eye area.

[0125] Based on the above embodiment providing a gaze depth acquisition method, the present disclosure also provides a gaze depth acquisition device. Fig.14 , including an eye region detection module 141, a pupil distance acquisition module 142 and a gaze depth acquisition module 143; wherein,

[0126] The eye region acquisition module 141 is used to acquire the eye region of the target object in the input image;

[0127] The pupil distance acquisition module 142 is used to acquire the center position of the pupil in the eye area and calculate the distance between the two pupils according to the center position;

[0128] The gaze depth acquisition module 143 is configured to acquire a first gaze depth of the target object according to a preset correspondence between the pupil distance and the gaze depth.

[0129] In one embodiment, the eye region acquisition module includes a face region acquisition unit and an eye region acquisition unit;

[0130] The facial region acquisition unit is used to acquire the facial region of the target object in the input image; the facial region includes a first number of facial feature points;

[0131] The eye region acquisition unit is used to extract a second number of eye feature points from the facial feature points, and determine the eye region of the target object according to the second number of eye feature points.

[0132] In one embodiment, the pupil distance acquisition module includes a pupil positioning unit and a distance calculation unit;

[0133] The pupil positioning unit is used to obtain a left eye area and a right eye area in the eye area, and determine the center positions of the pupils in the left eye area and the right eye area respectively;

[0134] The distance calculation unit is used to calculate the distance between the two pupils according to the center positions of the pupils in the left eye and the right eye.

[0135] In one embodiment, the gaze depth acquisition module includes a correspondence acquisition unit and a gaze depth acquisition unit;

[0136] The corresponding relationship acquisition unit is used to acquire the corresponding relationship between the preset pupil distance and the gaze depth;

[0137] The gaze depth acquisition unit is configured to acquire a first gaze depth of the target object according to a preset pupil distance and a corresponding relationship between the pupil distance and the gaze depth.

[0138] In one embodiment, the correspondence relationship acquisition unit includes a gaze depth acquisition subunit, a pupil distance acquisition subunit and a correspondence relationship acquisition subunit;

[0139] The gaze depth acquisition subunit is used to acquire the distance between the eye and the gazed movable device when the relative positions of the eye and the camera are fixed, and the distance is the gaze depth;

[0140] The pupil distance acquisition subunit is used to acquire the distance between the two pupils when the movable device being watched is stationary;

[0141] The correspondence acquisition subunit is used to perform polynomial fitting on multiple groups of gaze depths and distances to obtain an objective function of gaze depth and pupil distance, wherein the objective function is a preset correspondence between pupil distance and gaze depth.

[0142] In one embodiment, it also includes a gaze depth optimization module;

[0143] The gaze depth optimization module is used to optimize the first gaze depth to obtain a third gaze depth, where the third gaze depth is the gaze depth after the first gaze depth is optimized.

[0144] In one embodiment, the gaze depth optimization module includes an eye region scaling unit, a gaze depth acquisition unit, and a gaze depth optimization unit;

[0145] The eye region scaling unit is used to scale the eye region according to a preset scaling ratio to obtain a scaled eye region and output the scaled eye region to the interpupillary distance acquisition module;

[0146] The gaze depth acquisition unit is used to acquire a second gaze depth output by the gaze depth acquisition module;

[0147] The gaze depth optimization unit is configured to acquire a third gaze depth according to the first gaze depth and the second gaze depth, where the third gaze depth is a gaze depth after the first gaze depth is optimized.

[0148] In one embodiment, the eye region scaling unit includes a distance length acquisition subunit, a distance length fitting subunit and a scaling ratio acquisition subunit;

[0149] The distance length acquisition subunit is used to acquire the horizontal distance between the eye and the camera, and the length of the eye area when the relative positions of the camera and the movable device are fixed;

[0150] The distance length fitting subunit is used to perform polynomial fitting on multiple groups of horizontal distances and lengths to obtain an objective function on the horizontal distances and lengths;

[0151] The scaling ratio acquisition subunit is used to obtain a first length corresponding to the eye area when the eye is at a first distance from the camera and to obtain a second length corresponding to the eye area when the eye is at a second distance from the camera, and to calculate a ratio of the first length to the second length, which is used as the scaling ratio.

[0152] In one embodiment, the gaze depth optimization unit comprises:

[0153] A horizontal distance acquisition subunit, used for acquiring the horizontal distance between the eye and the camera at a second gaze depth;

[0154] An optimized depth acquisition subunit, used to acquire the sum of the second gaze depth and the horizontal distance to obtain an optimized gaze depth;

[0155] The third depth acquisition subunit is used to perform weighted average processing on the first gaze depth and the optimized gaze depth to acquire an average gaze depth, and the average gaze depth is the third gaze depth.

[0156] It is understandable that the device provided in the embodiment of the present disclosure corresponds to the above method. For the specific content, reference can be made to the content of each embodiment of the method, which will not be repeated here.

[0157] In an exemplary embodiment, there is also provided a display device, comprising:

[0158] processor;

[0159] a memory for storing a computer program executable by the processor;

[0160] The processor is configured to execute the computer program in the memory to implement the following Figure 1 The steps of the method.

[0161] In an exemplary embodiment, a computer readable storage medium is also provided, such as a memory including instructions, and the above executable computer program can be executed by a processor. The readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0162] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0163] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for acquiring gaze depth, characterized in that: include: Get the eye region of the target object in the input image; Acquire the center position of the pupil in the eye area and calculate the distance between the two pupils according to the center position; Based on a preset correspondence between the pupil distance and the gaze depth, acquiring a first gaze depth of the target object according to the distance; The method further comprises: Optimizing the first gaze depth to obtain a third gaze depth, where the third gaze depth is the gaze depth after the first gaze depth is optimized; Optimizing the first gaze depth to obtain a third gaze depth includes: Scaling the eye area according to a preset scaling ratio to obtain a scaled eye area, so as to obtain a second gaze depth according to the scaled eye area; Acquire a third gaze depth according to the first gaze depth and the second gaze depth; Scaling the eye region according to a preset scaling ratio to obtain a scaled eye region includes: When the relative positions of the camera and the movable device are fixed, the horizontal distance between the eye and the camera is obtained, and the length of the eye area is obtained; Perform polynomial fitting on multiple groups of horizontal distances and lengths to obtain the objective function of the horizontal distance and length; Obtain a first length of the eye region when the eye is at a first distance from the camera, and obtain a second length of the eye region when the eye is at a second distance from the camera, and calculate a ratio of the first length to the second length, which is used as a scaling ratio.

2. The method according to claim 1, characterized in that Get the eye area of ​​the target object in the input image, including: Acquire a facial region of a target object in an input image; the facial region includes a first number of facial feature points; A second number of eye feature points are extracted from the facial feature points, and an eye region of the target object is determined according to the second number of eye feature points.

3. The method according to claim 1, characterized in that Obtaining the center position of the pupil in the eye region and calculating the distance between the two pupils according to the center position, including: Acquire a left eye region and a right eye region in the eye region, and determine the center positions of the pupils in the left eye region and the right eye region respectively; The distance between the two pupils is calculated based on the center position of the pupils in the left and right eyes.

4. The method according to claim 1, characterized in that: Based on a preset correspondence between the pupil distance and the gaze depth, obtaining a first gaze depth of the target object according to the distance includes: Obtaining a correspondence between a preset pupil distance and a gaze depth; Based on a preset correspondence between the pupil distance and the gaze depth, a first gaze depth of the target object is acquired according to the distance.

5. The method according to claim 4, characterized in that Get the correspondence between the preset pupil distance and gaze depth, including: When the relative positions of the eye and the camera are fixed, the distance between the eye and the movable device being gazed is obtained, and the distance is the gaze depth; Obtain the distance between the two pupils when the mobile device being watched is stationary; Polynomial fitting is performed on multiple groups of gaze depths and distances to obtain an objective function of gaze depth and pupil distance, wherein the objective function is used as a corresponding relationship between a preset pupil distance and gaze depth.

6. The method according to claim 1, characterized in that Acquiring a third gaze depth according to the first gaze depth and the second gaze depth includes: Obtaining the horizontal distance between the eye and the camera at the second gaze depth; Obtaining the sum of the second gaze depth and the horizontal distance to obtain an optimized gaze depth; The first gaze depth and the optimized gaze depth are weighted averaged to obtain an average gaze depth, which is the third gaze depth.

7. A gaze depth acquisition device, characterized in that: It includes an eye area detection module, a pupil distance acquisition module and a gaze depth acquisition module; wherein, The eye region acquisition module is used to acquire the eye region of the target object in the input image; The pupil distance acquisition module is used to acquire the center position of the pupil in the eye area and calculate the distance between the two pupils according to the center position; The gaze depth acquisition module is used to acquire a first gaze depth of the target object according to a preset pupil distance and a corresponding relationship between the pupil distance and the gaze depth; The apparatus also includes a gaze depth optimization module; The gaze depth optimization module is used to optimize the first gaze depth to obtain a third gaze depth, where the third gaze depth is the gaze depth after the first gaze depth is optimized; The gaze depth optimization module includes an eye region scaling unit, a gaze depth acquisition unit and a gaze depth optimization unit; The eye region scaling unit is used to scale the eye region according to a preset scaling ratio to obtain a scaled eye region and output the scaled eye region to the interpupillary distance acquisition module; The gaze depth acquisition unit is used to acquire a second gaze depth output by the gaze depth acquisition module; The gaze depth optimization unit is configured to obtain a third gaze depth according to the first gaze depth and the second gaze depth, where the third gaze depth is a gaze depth after the first gaze depth is optimized; The eye region scaling unit includes a distance length acquisition subunit, a distance length fitting subunit and a scaling ratio acquisition subunit; The distance length acquisition subunit is used to acquire the horizontal distance between the eye and the camera, and the length of the eye area when the relative positions of the camera and the movable device are fixed; The distance length fitting subunit is used to perform polynomial fitting on multiple groups of horizontal distances and lengths to obtain an objective function on the horizontal distances and lengths; The scaling ratio acquisition subunit is used to obtain a first length corresponding to the eye area when the eye is at a first distance from the camera and to obtain a second length corresponding to the eye area when the eye is at a second distance from the camera, and to calculate a ratio of the first length to the second length, which is used as the scaling ratio.

8. The device according to claim 7, characterized in that The eye region acquisition module includes a face region acquisition unit and an eye region acquisition unit; The facial region acquisition unit is used to acquire the facial region of the target object in the input image; the facial region includes a first number of facial feature points; The eye region acquisition unit is used to extract a second number of eye feature points from the facial feature points, and determine the eye region of the target object according to the second number of eye feature points.

9. The device according to claim 7, characterized in that The pupil distance acquisition module includes a pupil positioning unit and a distance calculation unit; The pupil positioning unit is used to obtain a left eye area and a right eye area in the eye area, and determine the center positions of the pupils in the left eye area and the right eye area respectively; The distance calculation unit is used to calculate the distance between the two pupils according to the center positions of the pupils in the left eye and the right eye.

10. The device according to claim 7, characterized in that The gaze depth acquisition module includes a correspondence acquisition unit and a gaze depth acquisition unit; The corresponding relationship acquisition unit is used to acquire the corresponding relationship between the preset pupil distance and the gaze depth; The gaze depth acquisition unit is configured to acquire a first gaze depth of the target object according to a preset pupil distance and a corresponding relationship between the pupil distance and the gaze depth.

11. The device according to claim 10, characterized in that The correspondence relationship acquisition unit includes a gaze depth acquisition subunit, a pupil distance acquisition subunit and a correspondence relationship acquisition subunit; The gaze depth acquisition subunit is used to acquire the distance between the eye and the gazed movable device when the relative positions of the eye and the camera are fixed, and the distance is the gaze depth; The pupil distance acquisition subunit is used to acquire the distance between the two pupils when the movable device being watched is stationary; The correspondence acquisition subunit is used to perform polynomial fitting on multiple groups of gaze depths and distances to obtain an objective function of gaze depth and pupil distance, wherein the objective function is a preset correspondence between pupil distance and gaze depth.

12. The device according to claim 7, characterized in that The gaze depth optimization unit comprises: A horizontal distance acquisition subunit, used for acquiring the horizontal distance between the eye and the camera at a second gaze depth; An optimized depth acquisition subunit, used to acquire the sum of the second gaze depth and the horizontal distance to obtain an optimized gaze depth; The third depth acquisition subunit is used to perform weighted average processing on the first gaze depth and the optimized gaze depth to acquire an average gaze depth, and the average gaze depth is the third gaze depth.

13. A display device, characterized in that: include: processor; a memory for storing a computer program executable by the processor; The processor is configured to execute the computer program in the memory to implement the method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: When the executable computer program in the storage medium is executed by a processor, the method according to any one of claims 1 to 6 can be implemented.

Images