Line of sight estimation method, line of sight estimation device, and line of sight estimation procedure

Abstract

A viewing distance estimation method involves acquiring a first image of a person's face containing a viewing object, captured by a camera device; detecting the size of the person's iris from the first image; calculating a first value representing the number of pixels indicating the size of the detected iris; acquiring the resolution of the first image; calculating a third value representing the actual size corresponding to one pixel based on the first value and a second value representing the actual size of the iris (which is predetermined); estimating the viewing distance corresponding to the acquired resolution and the calculated third value based on relational information representing the relationship between the resolution, the third value, and the viewing distance; and outputting estimation information containing the estimated viewing distance.

Description

Technical Field

[0001] This invention relates to a technique for detecting visual distance, which represents the distance between an object and a person's eyes. Background Technology

[0002] The technology used to estimate the distance (visual distance) between a person's eyes and the object they are looking at is a fundamental technology for various processes, primarily based on estimating the person's state (e.g., eye fatigue). As a technology for estimating visual distance, the technology disclosed in Patent Document 1 is known, for example. Patent Document 1 discloses a technology that measures the distance between the user and the display using a rangefinder; if the measured distance is below a threshold, it automatically interferes with the image displayed on the display; if the measured distance is above the threshold, it automatically restores the image displayed on the display, thereby reducing the further development of myopia.

[0003] Furthermore, Patent Document 2 also discloses a technique that calculates the actual size of each unit pixel based on the number of pixels of the iris diameter contained in the image and the known actual size of the iris, which is unrelated to the human, and calculates the actual size of the object based on the calculated actual size of each unit pixel and the number of pixels of the object other than the iris contained in the image.

[0004] However, the technology disclosed in Patent Document 1 requires a dedicated rangefinder for measuring line-of-sight. Furthermore, the technology disclosed in Patent Document 2 does not measure line-of-sight. Therefore, in order to estimate line-of-sight without requiring a rangefinder and through a simple configuration, further improvements to the technologies of Patent Document 1 and Patent Document 2 are needed.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Publication No. 2019-168687

[0008] Patent Document 2: Japanese Patent Publication No. 2004-501463 Summary of the Invention

[0009] This invention was made to solve the above-mentioned problems, and its purpose is to provide a technology that can estimate line-of-sight distance without the use of a rangefinder, but with a simple structure.

[0010] One aspect of the present invention relates to a viewing distance estimation method, which is a viewing distance estimation method for a viewing distance estimation device that estimates the viewing distance representing the distance between an object and a person's eyes. The method involves the computer of the viewing distance estimation device: acquiring a first image of the face of the person viewing the object, captured by a camera device; detecting the size of the person's iris from the first image; calculating a first value representing the number of pixels indicating the detected iris size; acquiring the resolution of the first image; calculating a third value representing the actual size corresponding to one pixel based on the first value and a second value representing the actual size of the iris (which is predetermined); estimating the viewing distance corresponding to the acquired resolution and the calculated third value based on relational information representing the relationship between the resolution, the third value, and the viewing distance; and outputting estimation information containing the estimated viewing distance.

[0011] According to the present invention, the line-of-sight distance can be estimated through a simple configuration without the need for a rangefinder. Attached Figure Description

[0012] Figure 1 This is an external view of the line-of-sight estimation system according to the first embodiment of the present invention.

[0013] Figure 2 This is a block diagram illustrating an example of the overall configuration of the line-of-sight estimation system according to the first embodiment of the present invention.

[0014] Figure 3 This is a flowchart illustrating an example of the processing of the line-of-sight estimation device according to the first embodiment of the present invention.

[0015] Figure 4 This is a flowchart illustrating an example of iris detection processing.

[0016] Figure 5 This is an illustration of iris diameter.

[0017] Figure 6 This is a schematic diagram representing an example of a display screen shown on a monitor.

[0018] Figure 7 This is a diagram representing the facial area.

[0019] Figure 8 This is a schematic diagram showing the eye detection area.

[0020] Figure 9 This is a schematic diagram representing an example of a binary image.

[0021] Figure 10 This is a schematic diagram representing an example of a binary image that has undergone shrinkage processing.

[0022] Figure 11 This is a schematic diagram representing a local area.

[0023] Figure 12 This is an explanatory diagram of the estimated center location of the iris.

[0024] Figure 13 This is a schematic diagram of a binary image representing a black island region appearing within the pupil region.

[0025] Figure 14 This is a schematic diagram representing the binary image after the coating process.

[0026] Figure 15 This is a schematic diagram representing a binary image showing the detection of the left and right pixels of the pupil region.

[0027] Figure 16 This is a schematic diagram representing a binary image showing the detection of the upper and lower pixels of the pupil region.

[0028] Figure 17 This is a block diagram illustrating an example of the overall configuration of the line-of-sight estimation system according to the second embodiment of the present invention.

[0029] Figure 18 This is a flowchart illustrating an example of the processing of the line-of-sight estimation device according to the second embodiment.

[0030] Figure 19 This is an explanatory diagram of the process for detecting facial orientation.

[0031] Figure 20 This is a schematic diagram illustrating an example of the overall configuration of the line-of-sight estimation system according to the third embodiment of the present invention.

[0032] Figure 21 This is a flowchart illustrating an example of the processing of the line-of-sight estimation device according to the third embodiment of the present invention.

[0033] Figure 22 This is an explanatory diagram illustrating the process of calculating the convergence angle. Detailed Implementation

[0034] The process of obtaining the present invention

[0035] With the widespread use of digital devices, people are increasingly spending long periods of time looking at the displays of portable devices such as smartphones and tablets at close range. This increased exposure can lead to eye strain. Integrating features into portable devices that encourage attention, such as measuring the viewing distance between the display and the viewer's eyes and activating this feature when the eyes get too close to the screen, could help prevent eye fatigue.

[0036] However, in the aforementioned Patent Document 1, rangefinders, primarily infrared rangefinders and ultrasonic transducer transmitters, are used to measure line-of-sight distance. Therefore, the technology in Patent Document 1 requires either pre-assembling the rangefinder into the portable terminal device or installing an externally assembleable rangefinder into the portable terminal device, which leads to a larger and more complex configuration of the portable terminal device.

[0037] Moreover, in the aforementioned Patent Document 2, although the knowledge that the actual size of the iris diameter is constant is utilized, the object being measured is merely the actual size of the object contained within the image, and the viewing distance is not measured.

[0038] Here, we explore the case where the actual size of each pixel is measured based on the known actual size of the iris diameter and the number of pixels representing the iris diameter in the image when the person is photographed, and the viewing distance is calculated using the measured actual size of each pixel.

[0039] Image resolution varies depending on the type of camera device or camera mode. For example, if the number of pixels in the iris is represented by n pixels in a 1-megapixel image, then the number of pixels in the iris is represented by 2n pixels in a 2-megapixel image. Thus, the weight of each pixel relative to the actual size of the known iris diameter varies depending on the image resolution. Therefore, the actual size of each pixel calculated solely from the actual size of the known iris diameter and the number of pixels representing the iris diameter detected from the image cannot be used to infer the viewing distance.

[0040] Here, the inventors of the present invention have conducted a detailed study on the above-mentioned problems and arrived at the following conclusion: by utilizing the actual size of the known iris and the number of pixels of the iris size detected from the image, as well as the resolution of the image, the viewing distance can be estimated through a simple configuration without setting up a rangefinder. Thus, the following embodiments of the present invention were conceived.

[0041] One aspect of the present invention relates to a viewing distance estimation method, which is a viewing distance estimation method for a viewing distance estimation device that estimates the viewing distance representing the distance between an object and a person's eyes. The method involves the computer of the viewing distance estimation device: acquiring a first image of the face of the person viewing the object, captured by a camera device; detecting the size of the person's iris from the first image; calculating a first value representing the number of pixels indicating the detected iris size; acquiring the resolution of the first image; calculating a third value representing the actual size corresponding to one pixel based on the first value and a second value representing the actual size of the iris (which is predetermined); estimating the viewing distance corresponding to the acquired resolution and the calculated third value based on relational information representing the relationship between the resolution, the third value, and the viewing distance; and outputting estimation information containing the estimated viewing distance.

[0042] Based on this configuration, a third value representing the actual size of the iris detected from the first image is calculated, based on a first value representing the number of pixels and a second value representing the actual size of the human iris. Based on information representing the resolution and the relationship between the third value and the viewing distance, the viewing distance corresponding to the resolution of the acquired first image and the calculated third value is inferred.

[0043] Thus, in this configuration, the viewing distance is estimated using a third value representing the actual size corresponding to a pixel and the image resolution. Therefore, the present invention can estimate the viewing distance with a simple configuration without the need for a rangefinder.

[0044] Alternatively, the distance estimation method may be described in the form of a portable terminal device equipped with a camera and a display, where the object is the display and the first image is an image captured by the camera.

[0045] Based on this configuration, the viewing distance of a person viewing the display of a portable terminal device relative to the display can be estimated. Therefore, for example, if a person is too close to the display, they can be reminded to pay attention. This can help reduce eye strain.

[0046] In the aforementioned viewing distance estimation method, the orientation of the face may also be detected based on the first image, and the viewing distance may be corrected based on the detected orientation of the face.

[0047] In the first image, the number of pixels representing the size of the iris (a first value) decreases as the face turns away from a frontal orientation, and simultaneously, the estimated viewing distance becomes larger than the actual viewing distance. This makes it impossible to accurately estimate the viewing distance. According to this configuration, by correcting the viewing distance based on the face's orientation, the accuracy of the estimated viewing distance can be improved.

[0048] In the aforementioned method for estimating viewing distance, the viewing distance can also be corrected by multiplying it by a correction factor, which reduces the viewing distance as the face turns away from the frontal orientation.

[0049] According to this configuration, the viewing distance is corrected by multiplying it by a correction factor that causes the viewing distance to decrease as the face turns away from the frontal orientation. Therefore, the viewing distance can be accurately estimated regardless of the face's orientation.

[0050] In the aforementioned distance estimation method, when detecting the size of the iris, the following steps can be taken: a second image containing the person's eye region is generated from the first image; the second image is binarized to generate a third image, in which a first brightness value represents pixels with gray values ​​less than a threshold, and a second brightness value represents pixels with gray values ​​greater than the threshold; pixels with second brightness values ​​that appear in the third image within a first brightness region having the first brightness value and meet predetermined conditions are replaced with pixels with the first brightness value to generate a fourth image; and the iris is detected using the fourth image.

[0051] According to this configuration, pixels with a second brightness value that appear in the first brightness region of the third image and meet predetermined conditions are replaced with pixels with a first brightness value to generate a fourth image. Thus, pixels with the second brightness value appearing in the region corresponding to the pupil within the first brightness region are coated with the first brightness value. Furthermore, the size of the iris is detected using the fourth image, which is the coated binary image. As a result, the influence of external light and background reflected onto the cornea can be suppressed, and the iris can be detected with high accuracy.

[0052] In the aforementioned method for estimating visual distance, when detecting the size of the iris, the center positions of the irises of the person's left and right eyes can also be detected, and the convergence angle of the person's eyes can be calculated based on the detected center positions of the irises of the left and right eyes and the estimated visual distance.

[0053] Based on this structure, since the convergence angle of the eyes is calculated based on the central position of the iris of the left and right eyes and the viewing distance, it can provide information for judging eye diseases.

[0054] In the aforementioned method for estimating visual distance, when calculating the convergence angle: based on the first image, the midpoint between the inner corners of the eyes, representing the center between the inner corners of the person's eyes, is detected; a first distance from the midpoint between the inner corners of the eyes to the center position of the iris of the left eye and a second distance from the midpoint between the inner corners of the eyes to the center position of the iris of the right eye are calculated; a first half-convergence angle is calculated based on the first distance and the estimated visual distance, and a second half-convergence angle is calculated based on the second distance and the estimated visual distance; the sum of the first half-convergence angle and the second half-convergence angle is used as the convergence angle for calculation.

[0055] According to this configuration, based on the midpoint between the inner corners of the eyes detected in the first image, a first distance from the midpoint between the inner corners to the center of the iris of the left eye and a second distance from the midpoint between the inner corners to the center of the iris of the right eye are calculated. A first convergence half-angle is calculated based on the first distance and the estimated visual distance, and a second convergence half-angle is calculated based on the second distance and the estimated visual distance. The sum of the first and second convergence half-angles is used as the convergence angle for calculation. Therefore, the convergence angle can be calculated correctly.

[0056] In the aforementioned distance estimation method, the estimated information may also be overlaid on the first image.

[0057] According to this configuration, speculative information including the estimated viewing distance is overlaid on a first image containing a person's face, and the speculative information can be displayed in real time on the first image.

[0058] In the aforementioned distance estimation method, the estimation information superimposed on the first image may also include a measurement object representing the actual size of an object within the first image, generated based on the first value and the second value.

[0059] According to this configuration, since the object being measured is displayed within the first image, the actual size of the object within the first image can be represented.

[0060] In the aforementioned method for estimating visual distance, when detecting the size of the iris, the sizes of the irises of the left and right eyes are detected respectively; when estimating the visual distance, for the left and right eyes respectively, the appropriateness of the iris detection result is determined based on the detected iris size, and the visual distance is estimated only using the third value of the eye that is determined to be appropriate for the left and right eyes.

[0061] When a person blinks, or due to external light or background reflection onto the cornea, the detected iris size can sometimes be smaller than expected. In such cases, the estimated visual distance is larger than the actual visual distance, reducing the accuracy of the estimation. Based on this structure, the appropriateness of the iris detection results for the left and right eyes is determined based on the individual iris sizes of the left and right eyes, and the visual distance is estimated using only the third value from the eye deemed appropriate. Therefore, the correct visual distance can be estimated by considering that the detected iris size may be smaller than expected.

[0062] In the aforementioned method for estimating viewing distance, the relationship information can also be a regression equation that uses the resolution and the third value as explanatory variables and the viewing distance as the objective variable.

[0063] Based on this structure, the sight distance can be accurately predicted using regression equations.

[0064] Another aspect of the present invention relates to a viewing distance estimation device, which estimates the viewing distance representing the distance between an object and a person's eyes. The device includes: an image acquisition unit for acquiring a first image captured by a camera device, containing the face of the person viewing the object; an iris detection unit for detecting the size of the person's iris from the first image; a pixel count calculation unit for calculating a first value representing the number of pixels representing the detected iris size; a resolution acquisition unit for acquiring the resolution of the first image; an actual size calculation unit for calculating a third value representing the actual size corresponding to one pixel, based on the first value and a second value representing the actual size of the iris (which is predetermined); an estimation unit for estimating the viewing distance corresponding to the resolution acquired by the resolution acquisition unit and the third value calculated by the actual size calculation unit, based on relational information representing the relationship between the resolution, the third value, and the viewing distance; and an output unit for outputting estimation information containing the viewing distance estimated by the estimation unit.

[0065] Another aspect of the present invention relates to a viewing distance estimation program that enables a computer to function as a viewing distance estimation device for estimating the viewing distance representing the distance between an object and a person's eyes. The viewing distance estimation device includes: an image acquisition unit for acquiring a first image captured by a camera device, containing the face of the person viewing the object; an iris detection unit for detecting the size of the person's iris from the first image; a pixel count calculation unit for calculating a first value representing the number of pixels representing the detected iris size; a resolution acquisition unit for acquiring the resolution of the first image; an actual size calculation unit for calculating a third value representing the actual size corresponding to one pixel, based on the first value and a second value representing the actual size of the iris (which is predetermined); an estimation unit for estimating the viewing distance corresponding to the resolution acquired by the resolution acquisition unit and the third value calculated by the actual size calculation unit, based on relational information representing the relationship between the resolution, the third value, and the viewing distance; and an output unit for outputting estimation information containing the viewing distance estimated by the estimation unit.

[0066] Based on these components, the same technical effect as the aforementioned line-of-sight estimation method can be achieved.

[0067] This invention can also be implemented as a line-of-sight estimation method that operates through a line-of-sight detection program. Furthermore, needless to say, such a computer program can be distributed via computer-readable non-transitory recording media such as CD-ROMs or communication networks such as the Internet.

[0068] Furthermore, the embodiments described below are all specific examples of the present invention. The numerical values, shapes, constituent elements, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present invention. Moreover, among the constituent elements in the following embodiments, those not described in the independent claims representing the superior concept are described as arbitrary constituent elements. Furthermore, the contents of all embodiments can be combined arbitrarily.

[0069] First Implementation Method

[0070] Figure 1 This is an external view of the line-of-sight estimation system 100 according to the first embodiment of the present invention. The line-of-sight estimation system 100 is composed of a portable terminal device such as a smartphone or tablet computer. However, this is only one example, and the line-of-sight estimation system 100 can also be configured by appropriately combining a desktop computer or cloud server with a camera and a display.

[0071] The line-of-sight estimation system 100 includes a line-of-sight estimation device 1, a camera device 2, and a display 3. The line-of-sight estimation device 1 estimates the distance between the eyes of a person U1 captured by the camera device 2 and the display 3, i.e., the line-of-sight distance.

[0072] The camera device 2 consists of a camera assembled in a portable terminal device. The camera device 2 is a camera capable of capturing color visible light images at a specified frame rate.

[0073] Display 3 is composed of a display device such as a liquid crystal display or an organic EL (ElectroLuminescence) display device assembled in a portable terminal device. Display 3 displays an image of the face of person U1 captured by camera device 2. In addition, display 3 overlays the image of person U1's face with estimated viewing distance information estimated by viewing distance estimation device 1.

[0074] Figure 2 This is a block diagram illustrating an example of the overall configuration of the distance estimation system 100 according to the first embodiment of the present invention. The distance estimation device 1 includes a processor 10 and a memory 20. The processor 10 is, for example, a CPU (Central Processing Unit). The processor 10 includes an image acquisition unit 11, an iris detection unit 12, a pixel count calculation unit 13, a resolution acquisition unit 14, an actual size calculation unit 15, an estimation unit 16, and an output unit 17. The image acquisition unit 11 to the output unit 17 are implemented, for example, by having the processor 10 execute a distance estimation program.

[0075] The image acquisition unit 11 acquires a facial image containing the face of a person viewing an object, captured by the imaging device 2. The object is, for example, a display 3. The image acquisition unit 11 sequentially acquires facial images captured at a predetermined frame rate. The facial image is an example of a first image containing the face of a person U1 viewing an object.

[0076] The iris detection unit 12 detects the size of the iris of the person U1 from the face image acquired by the image acquisition unit 11. The size of the iris can be either the iris diameter or the iris radius. Hereinafter, the explanation will be based on the example of the iris diameter.

[0077] Specifically, the iris detection unit 12 detects a facial region containing the area of ​​a person's face from the facial image, and generates an eye detection region (an example from the second image) containing the left and right eye regions of the person U1 from the detected facial region. The eye detection region is, for example, rectangular.

[0078] Next, the iris detection unit 12 binarizes the eye detection region to generate a binary image (an example of the third image). In this binary image, pixels with grayscale values ​​less than a threshold are represented by a first brightness value, and pixels with grayscale values ​​above the threshold are represented by a second brightness value. Here, if the eye detection region is composed of a color image, the iris detection unit 12 can convert the eye detection region into a grayscale image, for example, with grayscale values ​​from 0 to 255, and perform binarization processing on the converted grayscale image. As the binarization processing, Otsu's binarization processing can be used, for example. The first brightness value is, for example, white, and the second brightness value is, for example, black. That is, in this embodiment, a binary image is generated where white represents darker parts and black represents bright parts. The brightness value of white is, for example, 255, and the brightness value of black is, for example, 0.

[0079] Next, the iris detection unit 12 performs a coating process on the binary image, replacing black pixels with white pixels. These black pixels are pixels that appear in a white region (first brightness region) composed of white pixels and meet predetermined conditions. Then, the iris detection unit 12 uses the coated binary image to detect the size of the iris. This generates a binary image where black islands appearing in the pupil region (hereinafter referred to as the pupil region) within the white region are coated with white pixels. The coated binary image is an example of the fourth image. Details of the coating process will be explained later.

[0080] Reference Figure 8 In this embodiment, the eye includes the sclera (white of the eye) and a pupil 110 that is circular when viewed from the front and is surrounded by the sclera 103. The pupil 110 includes a pupillary opening 101 that is circular when viewed from the front and an annular iris 102 surrounding the pupillary opening 101. In this embodiment, it is assumed that the center positions of the iris, the pupil 110, and the pupillary opening 101 are the same.

[0081] Reference Figure 2 The pixel count calculation unit 13 calculates a first value representing the number of pixels that represent the diameter of the iris detected by the iris detection unit 12.

[0082] The resolution acquisition unit 14 acquires the resolution of the face image. Here, the resolution acquisition unit 14 may also acquire the resolution of the display 3 pre-stored in the memory 20 as the resolution of the face image. Alternatively, the resolution acquisition unit 14 may acquire the resolution of the face image by acquiring the resolution corresponding to the shooting mode when the camera device 2 captures the face image. Alternatively, the resolution acquisition unit 14 may acquire the resolution of the face image by counting the resolutions of the face images acquired by the image acquisition unit 11. The acquired resolution may include, for example, the horizontal resolution, or both horizontal and vertical resolutions.

[0083] The actual size calculation unit 15 calculates a third value representing the actual size corresponding to one pixel, based on a first value calculated by the pixel count calculation unit 13 and a second value representing the actual size of the predetermined iris diameter. The actual size of the iris diameter is a constant actual size of the iris diameter independent of the human U1, and is a known value. For example, a value of about 12 mm is used as the second value. In addition, when the iris radius is used as the size of the iris, the actual size of the iris radius is used as the second value.

[0084] The estimation unit 16 estimates the viewing distance corresponding to the resolution obtained by the resolution acquisition unit 14 and the third value calculated by the actual size calculation unit 15, based on the relationship information representing the relationship between the resolution and the third value.

[0085] As relational information, for example, a regression equation pre-created by performing regression analysis on multiple datasets that correspond to resolution, tertiary value, and viewing distance can be used, in which viewing distance is the objective variable and resolution and tertiary value are the explanatory variables. Alternatively, as relational information, a machine learning model that has learned from multiple datasets using neural networks or similar methods can be used, in which viewing distance is the output and resolution and tertiary value are the inputs.

[0086] Data sets can be obtained through the following practical measurements. For example, using a camera device 2 with a certain resolution, the number of pixels in the iris diameter at each distance is measured while changing the distance between the camera device 2 and the person U1. Next, a third value is obtained by dividing the measured number of pixels in the iris diameter by the actual size of the iris diameter (e.g., 12 mm). This third value can be calculated multiple times while changing the resolution. Thus, multiple data sets corresponding to resolution, the third value, and viewing distance are obtained.

[0087] The output unit 17 outputs the estimation information including the line-of-sight, which is estimated by the estimation unit 16.

[0088] The memory 20 is composed of, for example, a non-volatile storage device that stores relational information.

[0089] Next, the processing of the line-of-sight estimation device 1 will be explained. Figure 3 This is a flowchart illustrating an example of the processing of the line-of-sight estimation device 1 according to the first embodiment of the present invention. Additionally, Figure 3 The flowchart shown is executed at a specified sampling period. The specified sampling period is, for example, the frame period of the camera device 2.

[0090] In step S1, the image acquisition unit 11 acquires a facial image from the imaging device 2. In step S2, the iris detection unit 12 performs iris detection processing to detect the iris diameter of the person U1 included in the facial image. Here, the iris detection unit 12 detects the iris diameters of the left and right eyes of the person U1. However, this is only one example; the iris detection unit 12 can also detect the iris diameter of either the left or right eye. Details of the iris detection processing will be provided later. Figure 4 The flowchart will be explained later.

[0091] In step S3, the pixel count calculation unit 13 calculates a first value by counting the number of pixels corresponding to the iris diameter detected by the iris detection unit 12. Here, the pixel count calculation unit 13 can calculate the first value by counting the number of pixels corresponding to the iris diameter of both the left and right eyes. Figure 5 This is an explanatory diagram of the iris diameter. During iris detection processing, the left-end pixel P6 and the right-end pixel P7 of the iris are detected. Therefore, the pixel count calculation unit 13 can count the number of pixels in the face image located between the left-end pixel P6 and the right-end pixel P7, and use the obtained pixel value as a first value for calculation. For example, the pixel count calculation unit 13 can also calculate the first value by counting the number of pixels at the X-coordinate located between the X-coordinate of the left-end pixel P6 and the X-coordinate of the right-end pixel P7. The X-coordinate represents the horizontal (horizontal) X-direction coordinate of the face image. The Y-coordinate represents the vertical (vertical) Y-direction coordinate of the face image.

[0092] In this example, the length of the iris in the X direction is used as the iris diameter. This is because the iris diameter in the X direction is less affected by the upper eyelid and blinking, and the iris diameter is generally taken as the length of the iris in the X direction. However, the invention is not limited to this; the iris diameter can also be the length of the iris in the Y direction (the length between the upper and lower pixels of the pupil region). Alternatively, the iris diameter can also be the length between the left pixel P6′ and the right pixel P7′ of the iris on the diagonal line L0 passing through the center position P0 of the iris.

[0093] In step S4, the resolution acquisition unit 14 acquires the resolution of the face image. Here, it is assumed that the horizontal resolution of the face image is acquired.

[0094] In step S5, the actual size calculation unit 15 calculates a third value by dividing the first value by the second value. Thus, a third value representing the actual size of each pixel within the face image is obtained. Here, the actual size calculation unit 15 can calculate the third value separately for the left and right eyes.

[0095] In step S6, the estimation unit 16 inputs the third value and resolution into the relational information to estimate the viewing distance. Here, the estimation unit 16 can input, for example, the average value of the third value calculated for the left and right eyes respectively into the relational information.

[0096] Alternatively, the estimation unit 16 can determine whether the left and right eyes represent appropriate iris diameters based on the iris diameters calculated by the iris detection processing, and estimate the visual distance using only the third value of the eye determined to be appropriate. Specifically, if the estimation unit 16 determines that both eyes represent appropriate iris diameters, it can input the average of the third values ​​of the left and right eyes into the relational information to estimate the visual distance. On the other hand, if only one eye represents an appropriate iris diameter, the estimation unit 16 can input only the third value of that appropriate eye into the relational information to estimate the visual distance. Furthermore, if neither eye represents an appropriate iris diameter, the estimation unit 16 may not estimate the visual distance. In this case, the estimation unit 16 can input an error signal to the output unit 17.

[0097] Whether the left and right eyes represent an appropriate iris diameter is determined, for example, as follows. First, the estimation unit 16 calculates the width of a predetermined portion of the face included in the face image. For example, the forehead can be used as a predetermined portion of the face. For instance, the estimation unit 16 performs feature point detection processing on the face image, detecting feature points on the left and right sides of the forehead. Then, the distance between the left and right feature points is calculated as the width of the forehead. Next, the estimation unit 16 multiplies the forehead width by a predetermined coefficient to calculate a diameter reference value. Then, if the iris diameter detected by the iris detection processing is below the diameter reference value, the estimation unit 16 can determine that the iris diameter is inappropriate.

[0098] Therefore, even if the detected iris size is smaller than expected due to factors such as blinking, external light, and background reflection on the cornea, the visual distance can still be accurately estimated.

[0099] In step S7, the output unit 17 generates and outputs prediction information containing the predicted viewing distance. For example, the output unit 17 may generate a display screen in which the prediction information is overlaid on a face image and display the display screen on the monitor 3.

[0100] Figure 6This is a schematic diagram illustrating an example of a display screen G1 shown on monitor 3. On display screen G1, a gauge object M1 is displayed above the eyes. The gauge object M1 comprises multiple scale lines M11. The gauge object M1 serves as a reference for representing the actual size of objects within the facial image. The gauge object M1 comprises, for example, ten scale lines M11. The spacing between adjacent scale lines M11 corresponds to actual dimensions. In this example, the spacing between scale lines M11 corresponds to 1 cm. Therefore, a person U1 viewing display screen G1 can grasp the actual size of any part within the facial image by observing the spacing between the scale lines M11. In this example, since the width of the lips corresponds to approximately four intervals of the scale lines M11, the width of the lips can be estimated to be approximately 4 cm.

[0101] The measurement object M1 is calculated as follows. First, the output unit 17 calculates the number of pixels per unit length by dividing the second pixel by the first pixel. Second, the output unit 17 calculates the number of pixels between the scale lines M11 in the face image by multiplying the number of pixels per unit length by the actual size value of the spacing of the scale lines M11 (1 cm in this example). Then, the output unit 17 generates the measurement object M1 by arranging the scale lines M11 for each calculated number of pixels.

[0102] A display bar R1 for displaying viewing distance is arranged at the top of the display screen G1. In this example, the viewing distance is recorded as ○0cm in the display bar R1. Thus, the user U1 operating the portable information terminal can know that the viewing distance up to the display screen 3 is ○0cm.

[0103] Here, the output unit 17 can also determine whether the estimated viewing distance is below the prescribed viewing distance reference value. If the estimated viewing distance is below the viewing distance reference value, a message urging attention is displayed on the display screen G1.

[0104] Additionally, in display screen G1, the iris center position P0 is overlaid on the center of the pupil. Furthermore, the positions of the upper eyelid P10, lower eyelid P11, outer corner of the eye P12, and inner corner of the eye P13 are overlaid on the face image of person U1. Furthermore, the circle L4 representing the outer edge of the iris is overlaid on the face image. Additionally, the rectangle L3 passing through the positions of the upper eyelid P10, outer corner of the eye P12, inner corner of the eye P13, and lower eyelid P11 is overlaid on the face image. This eye-related information utilizes the processing results of the iris detection process described later.

[0105] Therefore, the display screen G1 can display information related to the eyes of the person U1, primarily iris information, in real time on the facial image captured by the camera device 2.

[0106] Secondly, the details of iris detection and processing will be explained. Figure 4 This is a flowchart illustrating an example of iris detection processing. In step S41, the iris detection unit 12 inputs a face image into a classifier for detecting face regions, and detects the face regions. Figure 7 This is a schematic diagram representing the facial region 40. (Example) Figure 7 As shown, the iris detection unit 12 detects a rectangular area including the upper part of the forehead, the lower part of the chin, and the hairline of the ears as the face region 40. Here, although the face region 40 does not include all the hair, it could also include the entire hairline. Figure 7 Since the facial image is an image of person U1 taken from the front, it includes the left and right eyes.

[0107] In step S42, the iris detection unit 12 inputs the face region 40 detected in step S41 into a classifier for detecting the eye detection region, and detects the eye detection region. Figure 8 This is a schematic diagram representing the eye detection area 50. (Example) Figure 8 As shown, the eye detection area 50 encompasses the entire eye area and includes some extra rectangular regions to accommodate the eye's size. Figure 8 Extract the eye detection areas of the left and right eyes, each 50.

[0108] In step S43, the iris detection unit 12 converts the eye detection region 50 detected in step S42 into a grayscale image. As a grayscale image conversion process, one could employ, for example, a process that calculates the average grayscale values ​​of the red, green, and blue components of each pixel constituting the eye detection region 50. However, this is only one example, and other processes could also be used as the grayscale image conversion process.

[0109] In step S44, the iris detection unit 12 binarizes the eye detection region 50, which has been converted into a grayscale image, and generates a binary image 60. Figure 9 This is a schematic diagram representing an example of a binary image 60. In Figure 9 For example, in the eye detection region 50, a binary image 60 is generated, using white to represent dark areas such as the pupil and eyelashes, and black to represent bright areas such as the sclera and skin. Figure 9 For example, a white region D1 composed of white pixels represents information about the eye. Figure 9 In the example, due to the limited projection of external light and background onto the cornea, no black island areas appear in the white region D1. In cases where projection onto the cornea is present, such as... Figure 13 As shown, the black island region D2 appears.

[0110] In step S45, the iris detection unit 12 generates a binary image 70 by applying a narrowing process to the binary image 60 to remove unwanted black areas around the white area D1. Figure 10 This is a schematic diagram representing an example of a binary image 70 that has undergone shrinkage processing. Figure 10 For example, a bounding rectangle is defined for the white region D1 in the binary image 60, and the black region outside the bounding rectangle is removed to generate the binary image 70. Thus, removing double eyelids, dark circles under the eyes, moles around the eyes, and glasses around the eyes can improve the accuracy of subsequent processing.

[0111] In step S46, the iris detection unit 12 separates the binary image 70 into multiple local regions by separating each pixel in the X direction. Figure 11 This is a schematic diagram representing a local region 71. Figure 11 In this example, the iris detection unit 12 divides the binary image 70 into 10 equal parts horizontally. Thus, the binary image 70 is divided into ten local regions 71, each with a length along the Y direction. Here, the iris detection unit 12 divides the binary image 70 into ten local regions 71, but this is only one example. The number of divisions can also be an integer between 2 and 9, or an integer greater than or equal to 11.

[0112] In step S47, the iris detection unit 12 calculates the average brightness value of each of the ten local regions 71. Here, since the brightness value of white is 255 and the brightness value of black is 0, the average brightness value is calculated, for example, by the formula described below.

[0113] Average brightness value = Number of white pixels in local area 71 × 255 / Number of pixels in local area 71

[0114] In step S48, the iris detection unit 12 calculates the X coordinate of the iris's predicted center position. Figure 12 This is an explanatory diagram of the predicted iris center position P3. The predicted iris center position P3 is a predicted position of the iris center and differs from the final calculated iris center position P0. Due to the influence of double eyelids, eyelash density, and false eyelashes, these areas, as the white region D1, are sometimes over-represented. In such cases, the sclera 103 may be over-painted. To avoid this, the predicted iris center position P3 is calculated in this embodiment.

[0115] exist Figure 12In the example, the average brightness value of the fifth local region 71a from the left is the largest. Therefore, the iris detection unit 12 calculates the X-coordinate of the iris estimation center position P3 using the coordinates of the midpoint of the local region 71a in the X direction. However, depending on the width of the local region 71 in the X direction, it may be inappropriate to use the midpoint of the local region 71 in the X direction as the X-coordinate of the iris estimation center position P3. In this case, the left or right end of the local region 71 in the X direction can also be used as the X-coordinate of the iris estimation center position P3 for calculation.

[0116] In step S49, the iris detection unit 12 calculates the Y coordinate of the iris prediction center position P3. (Refer to...) Figure 12 The iris detection unit 12 detects the uppermost point P1 and the lowermost point P2 of the white pixel in the local region 71a, and calculates the Y-coordinate of the iris prediction center position P3 using the midpoint between the uppermost point P1 and the lowermost point P2. However, due to the influence of eyelashes and makeup, the uppermost point P1 and the lowermost point P2 may sometimes appear in the adjacent local region 71b on the left or the adjacent local region 71c on the right. Here, the iris detection unit 12 may also calculate the uppermost and lowermost points in each of the local regions 71a to 71c, average the three calculated uppermost points to obtain the average uppermost point, and average the three calculated lowermost points to obtain the average lowermost point, and calculate the Y-coordinate of the average uppermost point and average lowermost point as the midpoint of the average uppermost point and average lowermost point as the iris prediction center position P3.

[0117] In step S50, the iris detection unit 12 performs a coating process on the binary image 70. In visible light images, due to ambient brightness, external light and background may sometimes be reflected onto the cornea. In cases of significant reflection, brighter areas such as white may appear within the black or brown pupil. In such situations, if the eye image is binarized, black islands may appear within the pupil area, making it impossible to detect iris information with high accuracy. Therefore, in this embodiment, a coating process is performed.

[0118] Figure 13 This is a schematic diagram of a binary image 70 showing a black island region D2 within the pupil region. Figure 13 The left image shows the binary image 70 for the left eye, and the right image shows the binary image 70 for the right eye. Figure 13 As shown, it can be seen that in both the left and right binary images 70, black island regions D2 are scattered within the white region D1, which corresponds to the pupil appearing in the center. Applying the black island regions D2 is a coating process. In this specification, when viewing person U1 from the front, the eye located on the left is the left eye, and the eye located on the right is the right eye. However, this is only an example, and this relationship can also be reversed.

[0119] The detailed process of coating is as follows. First, the iris detection unit 12 sets a vertical line L1 parallel to the Y direction at the X coordinate of the binary image 70 at the iris estimation center position P3. Second, the iris detection unit 12 detects the first white pixel appearing on the vertical line L1 from the top of the binary image 70 as the upper pixel P4. Next, the iris detection unit 12 detects the first white pixel appearing on the vertical line L1 from the bottom of the binary image 70 as the lower pixel P5. Next, the iris detection unit 12 determines whether the distance between the upper pixel P4 and the lower pixel P5 is greater than a first reference distance. Next, if the iris detection unit 12 determines that the distance between the upper pixel P4 and the lower pixel P5 is greater than the first reference distance, the iris detection unit 12 determines the black pixel located on the vertical line L1 between the upper pixel P4 and the lower pixel P5 as a black pixel that meets the specified conditions, and replaces the black pixel with a white pixel. On the other hand, the iris detection unit 12 does not perform the replacement on the vertical line L1 if it determines that the distance between the upper pixel P4 and the lower pixel P5 is less than a first reference distance. The first reference distance is, for example, an appropriate distance based on an assumed iris diameter.

[0120] The iris detection unit 12 performs a coating process on each longitudinal line L1 within a range of a left reference distance from the estimated iris center position P3 in the X direction, and performs the same coating process on each longitudinal line L1 within a range of a right reference distance from the estimated iris center position P3 in the X direction. The sum of the left and right reference distance ranges is used as a second reference distance. The left and right reference distance ranges are, for example, the same range. The second reference distance is, for example, a distance slightly larger than the assumed iris diameter. Therefore, the coating process can be applied specifically to the longitudinal lines L1 located in the pupil region.

[0121] Figure 14 This is a schematic diagram representing the binary image 80 after the coating process. Figure 14 The left figure represents the... Figure 13 The binary image 70 on the left is used to apply the coating process to the binary image 80. Figure 14 The right figure represents the... Figure 13 The binary image 70 on the right is used to apply the coating process to the binary image 80. For example... Figure 14 As shown, it can be seen that in Figure 13 The existing black island region D2 is painted with white pixels, generating a white region D3 composed of a single white pixel. On the other hand, it can be seen that the black island region located in the eyelash area is not painted. That is, it can be seen that the vertical line L1 located in the pupil region is painted with particular emphasis.

[0122] In step S51, the iris detection unit 12 detects the left-end pixel P6 and the right-end pixel P7 of the pupil region, respectively. Figure 15This is a schematic diagram of a binary image 80 showing the detection of the left-end pixel P6 and the right-end pixel P7 of the pupil region. The iris detection unit 12, in the white region D3 of the binary image 80, investigates the change in brightness value pixel by pixel in the X direction, starting from the iris prediction center position P3. Furthermore, the iris detection unit 12 detects the black pixel initially appearing on the left side in the X direction as the left-end pixel P6, and the black pixel initially appearing on the right side in the X direction as the right-end pixel P7.

[0123] In step S52, the iris detection unit 12 calculates the X coordinate of the iris center position P0 by taking the midpoint between the left pixel P6 and the right pixel P7 as the midpoint.

[0124] In step S53, the iris detection unit 12 detects the upper and lower pixels of the pupil region respectively. Figure 16 This is a schematic diagram of a binary image 80 showing the detection of the upper pixel P8 and the lower pixel P9 of the pupil region. The iris detection unit 12, in the white region D3 of the binary image 80, investigates the change in brightness value pixel by pixel in the Y direction, starting from the X coordinate of the iris center position P0. Furthermore, the iris detection unit 12 detects the black pixel that initially appears on the upper side in the Y direction as the upper pixel P8, and the black pixel that initially appears on the lower side in the Y direction as the lower pixel P9.

[0125] In step S54, the iris detection unit 12 calculates the Y-coordinate of the iris center position P0 using the midpoint between the upper pixel P8 and the lower pixel P9 as the coordinate. Through this process, the iris center position P0 is calculated.

[0126] In step S55, the iris detection unit 12 calculates the distance between the left pixel P6 and the right pixel P7 as the iris diameter. For example, the iris detection unit 12 may also calculate the iris diameter using the difference between the X coordinate of the left pixel P6 and the X coordinate of the right pixel P7.

[0127] In addition, the iris detection unit 12 can also calculate separately. Figure 6The diagram shows the positions of the upper eyelid (P10), lower eyelid (P11), outer corner of the eye (P12), inner corner of the eye (P13), circle (L4), and rectangle (L3). The upper eyelid position P10 can be the upper pixel of the edge of the white region obtained by performing morphological gradient calculation on the binary image 80. The lower eyelid position P11 can be the lower pixel of the edge of the white region obtained by performing morphological gradient calculation on the binary image 80. The outer corner of the eye position P12 can be the left pixel of the white region in the binary image 80 of the left eye. The inner corner of the eye position P13 can be the right pixel of the white region in the binary image 80 of the left eye. Circle L4 can be a circle with the diameter of the iris centered at the iris center position P0. Rectangle L3 can be a rectangle formed by the positions of the upper eyelid (P10), lower eyelid (P11), outer corner of the eye (P12), and inner corner of the eye (P13). If the processing in step S55 is complete, the process proceeds to... Figure 3 Step S3.

[0128] Thus, in this embodiment, based on a first value representing the number of pixels as the iris diameter detected from the first image and a second value representing the actual size of the human iris, a third value representing the actual size corresponding to one pixel is calculated. Based on information representing the resolution of the image and the relationship between the third value and the viewing distance, the viewing distance corresponding to the resolution of the face image and the calculated third value is estimated. Thus, in this embodiment, the viewing distance is estimated using the third value representing the actual size corresponding to one pixel and the resolution of the face image. Therefore, in this embodiment, the viewing distance can be estimated using only a simple configuration, even without setting up a rangefinder.

[0129] Second Implementation Method

[0130] The second implementation method is an implementation method that corrects the viewing distance based on the orientation of the face. Figure 17 This is a block diagram illustrating an example of the overall configuration of the sight distance estimation system 100 according to the second embodiment of the present invention. Furthermore, in this embodiment, the same reference numerals are used for the same constituent elements as in the first embodiment, and their descriptions are omitted. The sight distance estimation system 100 of the second embodiment includes a sight distance estimation device 1A. The processor 10A of the sight distance estimation device 1A further includes a sight distance correction unit 18 relative to the processor 10 of the sight distance estimation device 1.

[0131] The viewing distance correction unit 18 detects a face orientation degree, representing the degree of the face's lateral orientation, based on the face image, and corrects the viewing distance estimated by the estimation unit 16 based on the detected face orientation degree. Specifically, the viewing distance correction unit 18 corrects the viewing distance by multiplying it by a correction coefficient that reduces the viewing distance as the face orientation degree changes from a frontal orientation to a direction away.

[0132] Figure 18 This is a flowchart illustrating an example of the processing of the line-of-sight estimation device 1A according to the second embodiment. In this process, for... Figure 3 The same processing is assigned the same processing number.

[0133] In step S1801, following step S6, the viewing distance correction unit 18 detects the facial orientation of person U1 based on the facial image. Figure 19 This is an explanatory diagram of the process for detecting facial orientation.

[0134] First, the viewing distance correction unit 18 detects facial feature points by applying a landmark process to the facial image. Feature points, also known as markers, represent facial features such as the tip of the nose, the edges of the lips, and the curvature points of facial lines.

[0135] Next, the viewing distance correction unit 18 sets a vertical center line 131 and a horizontal center line 132 based on feature points 9X set on the face in the face region 40. For example, the viewing distance correction unit 18 can set a straight line passing through feature point 133 representing the center of the bridge of the nose and parallel to the vertical edge of the face region 40 as the vertical center line 131. Feature point 133 is, for example, the third feature point 9X from the top among the five feature points 9X representing the bridge of the nose. Next, the viewing distance correction unit 18 sets a straight line, for example, passing through feature point 133 and parallel to the horizontal edge of the face region 40 as the horizontal center line 132. In addition, the vertical center line 131 and the horizontal center line 132 have been described using feature point 133 passing through the center of the bridge of the nose as an example. However, for example, they can also be set to pass through feature point 134 at the lower end of the bridge of the nose, or they can be set to pass through feature point 135 at the upper end of the bridge of the nose.

[0136] Next, the viewing distance correction unit 18 divides the horizontal center line 132 using feature points 133 and calculates the lengths of the right interval K1 and the left interval K2. Then, the viewing distance correction unit 18 calculates the ratio of the right interval K1 to the left interval K2 when the length of the horizontal center line 132 is 100%, and calculates the face orientation based on this ratio. If the ratio of the right interval K1 is K1 and the ratio of the left interval K2 is K2, the face orientation can be calculated, for example, by -(K1-K2). In this formula, the initial negative number is used to make the face orientation positive when facing right. For example, if K1 = 30% and K2 = 70%, the face orientation is -(30-70) = 40. For example, if K1 = 70% and K2 = 30%, the face orientation is -(70-30) = -40. For example, if K1 = 50% and K2 = 50%, the face orientation is -(50-50) = 0.

[0137] Therefore, as facial orientation increases in the positive direction, it indicates that the face is facing more to the right, and as facial orientation increases in the negative direction, it indicates that the face is facing more to the left. Moreover, when facial orientation is 0, it indicates that the face is facing forward.

[0138] In step S1802, the viewing distance correction unit 18, referring to pre-created correction coefficient calculation information, determines a correction coefficient corresponding to the calculated absolute value of the face orientation, and corrects the viewing distance by multiplying the viewing distance estimated in step S6 by the determined correction coefficient. The coefficient calculation information is information that ensures the absolute value of the face orientation corresponds to the correction coefficient in a manner that reduces the correction coefficient within a range from 0 to 1 as the absolute value of the face orientation increases. For example, in the coefficient calculation information, a correction coefficient of the maximum value 1 corresponds to a face orientation of 0. Furthermore, the coefficient calculation information ensures that the absolute value of the face orientation corresponds to the correction coefficient in a manner that makes the correction coefficient approach a predetermined lower limit value less than 1 as the absolute value of the face orientation approaches the maximum value 50.

[0139] In step S7, the output contains the predicted sight distance information that has been corrected.

[0140] As the face turns away from a frontal orientation, the number of pixels representing the iris diameter in the face image (a first value) decreases, and simultaneously, the estimated viewing distance becomes larger than the actual viewing distance. This makes it impossible to accurately estimate the viewing distance. According to this configuration, because the viewing distance is corrected based on the face's orientation, the accuracy of the estimated viewing distance can be improved.

[0141] Third Implementation Method

[0142] The third implementation method is the method of calculating the convergence angle. The convergence angle refers to the angle at which the line of sight of both eyes can converge to the position of the object being viewed. Figure 20 This is a schematic diagram illustrating an example of the overall configuration of the sight distance estimation system 100 according to the third embodiment of the present invention. Furthermore, in this embodiment, the same reference numerals are used for the same constituent elements as in the first and second embodiments, and their descriptions are omitted. The sight distance estimation system 100 of the third embodiment includes a sight distance estimation device 1B. The processor 10B of the sight distance estimation device 1B further includes a convergence angle calculation unit 19 relative to the processor 10A of the sight distance estimation device 1A.

[0143] The convergence angle calculation unit 19 calculates the convergence angle of the eyes of the person U1 based on the center positions of the irises of the left and right eyes of the person U1 detected by the iris detection unit 12 and the viewing distance corrected by the viewing distance correction unit 18.

[0144] Figure 21 This is a flowchart illustrating an example of the processing of the line-of-sight estimation device 1B according to the third embodiment of the present invention. In this process, for... Figure 18 The same processing is assigned the same processing number. Figure 22 This is an explanatory diagram of the process for calculating the convergence angle θ. In step S1901, following step S1802, the convergence angle calculation unit 19 calculates the distance between the iris center position P0_L of the left eye and the iris center position P0_R of the right eye, i.e., the iris center distance. Here, the iris center position P0_L and the iris center position P0_R are based on the iris center position P0 calculated in the iris detection process.

[0145] In step S1902, the convergence angle calculation unit 19 calculates the midpoint between the inner corner positions P13 of the left and right eyes, i.e., the midpoint between the inner corners P221. The positions P13 of the inner corners of the left and right eyes are calculated using the inner corner positions P13 obtained during iris detection processing. For example, the convergence angle calculation unit 19 can calculate the difference in the X-coordinates of the inner corner positions P13 of the left and right eyes respectively, and use the midpoint of the calculated difference as the midpoint between the inner corners P221 for calculation.

[0146] In step S1903, the convergence angle calculation unit 19 calculates the distance D221 (first distance) between the midpoint P221 between the inner canthi and the center position P0-L of the iris, and calculates the distance D222 (second distance) between the midpoint P221 between the inner canthi and the center position P0-R of the iris. For example, the convergence angle calculation unit 19 calculates the distance D221 as the difference between the X coordinate of the midpoint P221 between the inner canthi and the X coordinate of the center position P0-L of the iris, and calculates the distance D222 as the difference between the X coordinate of the midpoint P221 between the inner canthi and the X coordinate of the center position P0-R of the iris.

[0147] In step S1904, the convergence angle calculation unit 19 calculates the convergence half angle θ1 (first convergence half angle) of the left eye using the viewing distance L22 and the distance D221, and calculates the convergence half angle θ2 (second convergence half angle) using the viewing distance L22 and the distance D222.

[0148] Here, the convergence half-angle θ1 is calculated using arctan(D221 / L22), and the convergence half-angle θ2 is calculated using arctan(D222 / L22).

[0149] In step S1905, the convergence angle calculation unit 19 calculates the sum of the convergence half-angle θ1 and the convergence half-angle θ2 as the convergence angle θ.

[0150] The calculated convergence angle θ is included in the inferred information and output (step S7). Here, the convergence angle θ can also be displayed on the display screen G1, for example. Thus, the convergence angle θ can be indicated, and diagnostic materials for eye diseases can be provided.

[0151] The present invention can be modified in the following ways.

[0152] (1) The viewing distance estimation device 1 may also be configured independently of the display 3 and the camera device 2. In this case, the viewing distance estimated using the relationship information becomes the viewing distance between the camera device 2 and the person U1. Here, the estimation unit 16 can calculate the viewing distance between the display 3 and the person U1 by correcting the viewing distance estimated using the relationship information with information representing the relative positional relationship between the display 3 and the camera device 2.

[0153] (2) In the second embodiment, the face orientation is calculated by image processing, but the present invention is not limited to this, and the value input by the user using an operation device (e.g., a touch panel) without illustration can also be used.

[0154] (3) The iris detection process described in the first embodiment is only one example, and other iris detection processes can also be used in this invention. As an example of other iris detection processes, for example, the process using the Doug Leman algorithm can be cited.

[0155] (4) In the first to third embodiments, the first brightness value of the binary images 60, 70 and 80 is white and the second brightness value is black. However, the present invention is not limited to this and the first brightness value can also be black and the second brightness value can be white.

[0156] (5) Relationship information can also be formed by a lookup table that represents the relationship between resolution and third value and viewing distance.

[0157] (6) The third embodiment can also be applied to the first embodiment. In this case, the convergence angle is calculated using the uncorrected line of sight instead of the corrected line of sight.

[0158] Industrial availability

[0159] According to the present invention, since the estimated viewing distance can be constructed in a simple manner, it has a wider range of applications in the field of estimated viewing distance.

Claims

1. A method for estimating visual distance, which is a method for estimating visual distance using a visual distance estimation device that represents the distance between an object and a person's eyes, characterized in that, The computer of the line-of-sight estimation device performs the following steps: Acquire a first image of the face of the person viewing the object, captured by a camera device; Detect the size of the person's iris from the first image; Calculate a first value representing the number of pixels indicating the size of the detected iris; Obtain the resolution of the first image; Based on the first value and a second value representing the actual size of the iris (which is defined in advance), a third value representing the actual size corresponding to one pixel is calculated. Based on the relationship information representing the relationship between the resolution and the third value and the viewing distance, the viewing distance corresponding to the obtained resolution and the calculated third value is inferred; The output contains the inferred line-of-sight information. The orientation of the face is also detected based on the first image, and the viewing distance is corrected according to the detected orientation of the face.

2. The line-of-sight estimation method according to claim 1, characterized in that, The line-of-sight estimation device is a portable terminal device equipped with a camera and a display. The object is the display. The first image is an image captured by the camera.

3. The line-of-sight estimation method according to claim 1, characterized in that, When correcting the viewing distance, the viewing distance is corrected by multiplying it by a correction factor, which reduces the viewing distance as the face turns away from the frontal orientation.

4. The line-of-sight estimation method according to any one of claims 1 to 3, characterized in that, When detecting the size of the iris: Generate a second image containing the person's eye region from the first image; The second image is binarized to generate a third image. In the third image, a first brightness value is used to represent pixels with gray values ​​less than a threshold, and a second brightness value is used to represent pixels with gray values ​​greater than the threshold. A fourth image is generated by replacing pixels with the first brightness value that appear in the third image within a first brightness region having the first brightness value and that meet a predetermined condition with pixels of the first brightness value. The iris is detected using the fourth image.

5. The line-of-sight estimation method according to any one of claims 1 to 3, characterized in that, When measuring the size of the iris, the center positions of the irises of the person's left and right eyes are measured. The convergence angle of the person's eyes is calculated based on the detected center positions of the irises of the left and right eyes and the estimated visual distance.

6. The line-of-sight estimation method according to claim 5, characterized in that, When calculating the convergence angle: Based on the first image, the midpoint between the inner corners of the eyes, representing the center between the inner corners of the person's eyes, is detected; Calculate a first distance from the midpoint between the inner corners of the eyes to the center of the iris of the left eye and a second distance from the midpoint between the inner corners of the eyes to the center of the iris of the right eye; Calculate the first convergence half-angle based on the first distance and the inferred line-of-sight distance, and calculate the second convergence half-angle based on the second distance and the inferred line-of-sight distance; The sum of the first convergence half-angle and the second convergence half-angle is used as the convergence angle for calculation.

7. The line-of-sight estimation method according to any one of claims 1 to 3, characterized in that, The inferred information is also overlaid on the first image.

8. The line-of-sight estimation method according to any one of claims 1 to 3, characterized in that, The speculative information, which is overlaid on the first image, includes a measurement object representing the actual size of an object within the first image, generated based on the first and second values.

9. The line-of-sight estimation method according to any one of claims 1 to 3, characterized in that, When detecting the size of the iris, the size of the iris of both the left and right eyes is detected. When estimating the visual distance, for both the left and right eyes, the appropriateness of the iris detection result is determined based on the detected iris size, and the visual distance is estimated using only the third value of the eye that is determined to be appropriate for the left and right eyes.

10. The line-of-sight estimation method according to any one of claims 1 to 3, characterized in that, The relationship information is a regression equation that uses the resolution and the third value as explanatory variables and the viewing distance as the objective variable.

11. A distance estimation device, which estimates the distance between an object and a person's eyes, characterized in that... include: An image acquisition unit is used to acquire a first image of the face of the person viewing the object, captured by a camera device. The iris detection unit detects the size of the person's iris from the first image; A pixel count calculation unit is used to calculate a first value representing the number of pixels indicating the size of the detected iris; The resolution acquisition unit acquires the resolution of the first image; The actual size calculation unit calculates a third value representing the actual size corresponding to one pixel, based on the first value and a second value representing the actual size of the iris that is predetermined. The estimation unit, based on relational information representing the relationship between the resolution and the third value and the viewing distance, estimates the viewing distance corresponding to the resolution obtained by the resolution acquisition unit and the third value calculated by the actual size calculation unit; The output unit outputs prediction information containing the line-of-sight distance predicted by the prediction unit; as well as, The viewing distance correction unit detects the orientation of the face based on the first image and corrects the viewing distance according to the detected orientation of the face.

12. A computer program product comprising a line-of-sight estimation program, characterized in that, The distance estimation program enables a computer to function as a distance estimation device that estimates the distance between an object and a person's eyes, the distance estimation device comprising: An image acquisition unit is used to acquire a first image of the face of the person viewing the object, captured by a camera device. The iris detection unit detects the size of the person's iris from the first image; A pixel count calculation unit is used to calculate a first value representing the number of pixels indicating the size of the detected iris; The resolution acquisition unit acquires the resolution of the first image; The actual size calculation unit calculates a third value representing the actual size corresponding to one pixel, based on the first value and a second value representing the actual size of the iris that is predetermined. The estimation unit, based on relational information representing the relationship between the resolution and the third value and the viewing distance, estimates the viewing distance corresponding to the resolution obtained by the resolution acquisition unit and the third value calculated by the actual size calculation unit; The output unit outputs prediction information containing the line-of-sight distance predicted by the prediction unit; as well as, The viewing distance correction unit detects the orientation of the face based on the first image and corrects the viewing distance according to the detected orientation of the face.

Images