Information processing system, information processing method, and program

Abstract

To allow refractive power to be measured easily.SOLUTION: An information processing system includes: a learning model storage section for storing a learning model in which an image obtained by photographing eyes in a state in which light is emitted and is applied to a cornea, and the eyes' refractive index are learned by machine learning; an image acquisition section for acquiring a photographed image obtained by photographing a user's eyes by allowing a user terminal to emit light; and a determination processing section for estimating a user's refractive index by giving an acquired, photographed image to a learning model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] In Patent Document 1, the refractive power of the eye is measured using photorefraction. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent No. 9,408,535 Summary of the Invention [Problem to be solved by the invention]

[0004] To provide a technique for measuring eye refractive power more easily than with conventional eye refractive power measuring devices. [Means for solving the problem]

[0005] The main invention of the present invention for solving the above problem is an information processing system comprising: a learning model memory unit that stores a learning model that has been trained through machine learning from an image of an eye photographed with a light emitted and irradiated onto the cornea and the refractive value of the eye; an image acquisition unit that acquires an image of a user's eye photographed by a user terminal emitting a light; and an estimation processing unit that provides the acquired image to the learning model to estimate the refractive value of the user.

[0006] Other problems and solutions disclosed in this application will be made clear in the description of the preferred embodiments of the invention and the drawings. Effect of the Invention

[0007] According to the present invention, the refractive power can be easily measured. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing an example of the overall configuration of a refractive power measurement system which is an information processing system according to an embodiment of the present invention; [Diagram 2] FIG. 4 is a diagram illustrating a measurement process in the present embodiment. [Diagram 3] 1 is a diagram illustrating a user's eye 20 included in a captured image. [Figure 4] 2 is a diagram illustrating an example of a hardware configuration of a management server 2. FIG. [Diagram 5] 2 illustrates an example of the software configuration of a management server 2. FIG. [Figure 6] FIG. 11 is a diagram illustrating the flow of a learning process. [Figure 7] 11 is a diagram illustrating a flow of a refraction value prediction process. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] <Summary of the Invention> The present invention will be described below with reference to the preferred embodiments thereof. [Item 1] A learning model storage unit that stores a learning model that is learned by machine learning from an image of an eye taken with a light emitted and irradiated onto a cornea and a refractive value of the eye; an image acquisition unit that acquires an image of the user's eyes by emitting a light from the user terminal; an estimation processing unit that estimates a refractive index of the user by providing the acquired photographed image to the learning model; An information processing system comprising: [Item 2] The information processing system according to item 1, The refractive value includes at least one of a spherical power, a cylindrical power, and an axis of the cylindrical power; An information processing system comprising: [Item 3] The information processing system according to item 1, The learning model is a model that has learned the light area and the refraction value included in the image, A light extraction unit extracts a light area in which the light is captured from the captured image, the estimation processing unit estimates the refraction value by providing the light region to the learning model; An information processing system comprising: [Item 4] The information processing system according to item 1, The learning model is a model that has been trained to learn the area of ​​the cornea and the refractive value included in the image captured so that the light falls on the cornea, a cornea extraction unit that extracts a cornea area from the captured image, the estimation processing unit estimates the refractive value by providing the corneal region to the learning model; An information processing system comprising: [Item 5] A computer that stores a learning model that learns an image of a human eye captured by emitting light and a refractive index of the eye through machine learning, A step of acquiring a captured image of the user's eyes by emitting a light from the user terminal; A step of estimating a refractive index of the user by providing the acquired captured image to the learning model; 2. An information processing method comprising: [Item 6] A computer stores a learning model that learns an image of a human eye taken by emitting light and a refractive value of the eye by machine learning. A step of acquiring a captured image of the user's eyes by emitting a light from the user terminal; A step of estimating a refractive index of the user by providing the acquired captured image to the learning model; A program for executing the above.

[0010] <System Overview> 1 is a diagram showing an example of the overall configuration of a refractive power measurement system which is an information processing system according to one embodiment of the present invention. The refractive power measurement system of this embodiment is configured to include a management server 2. The management server 2 is communicably connected to a user terminal 1 via a communication network. The communication network is, for example, the Internet, and is constructed by a public telephone line network, a mobile phone line network, a wireless communication path, Ethernet (registered trademark), or the like.

[0011] The user terminal 1 is a computer operated by a user. The user terminal 1 can be, for example, a smartphone, a tablet computer, a personal computer, etc. In this embodiment, it is assumed that the user terminal 1 is a smartphone or a tablet computer, and is equipped with a camera (not shown) and a flashlight (not shown).

[0012] The management server 2 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0013] The refractive power measurement system of this embodiment estimates the refractive power of the user's eyes. Fig. 2 is a diagram for explaining the measurement in this embodiment. In the refractive power measurement system of this embodiment, a flashlight of the user terminal 1 is turned on so that the light from the flashlight is irradiated onto the cornea (iris of the eye), and the camera of the user terminal 1 captures an image of the user (which may be the upper body, the face, or just the eyes) so that the captured image includes light reflected by the cornea.

[0014] Fig. 3 is a diagram illustrating a user's eye 20 included in a captured image. Reflected light 22 of a flashlight is captured on a cornea 21 inside the eye 20. It is assumed that the shape of the flashlight is circular. The shape of this reflected light 22 may be deformed into an ellipse as shown in Fig. 3. The specific deformation varies depending on the refraction value.

[0015] In the refractive power measurement system of the present embodiment, a value indicating the refractive power of the user's eye (refractive value) is measured by a general measuring device. In the general measuring device, for example, a light is irradiated by a photorefraction method, and the light reflected from the fundus is detected to measure the refractive index. The refractive value measured by the general measuring device and an image of the reflected light from the cornea 21 are learned by machine learning to create a learning model. The image of the reflected light may be an image of the user's whole body, an image of the user's upper body, an image of the user's face, or an image of the user's eye. The image of the reflected light may be an image of the user with the eye or cornea extracted from the image of the user. In the refractive power measurement system, the user's refractive value can be estimated by providing the learning model with an image of the user photographed while the user is pointing a flashlight at his or her eye and emitting the light.

[0016] <Administration Server> FIG. 4 is a diagram showing an example of the hardware configuration of the management server 2. Note that the illustrated configuration is an example, and other configurations may be used. The management server 2 includes a CPU 201, a memory 202, a storage device 203, a communication interface 204, an input device 205, and an output device 206. The storage device 203 is, for example, a hard disk drive, a solid state drive, or a flash memory that stores various data and programs. The communication interface 204 is an interface for connecting to a communication network, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone line network, a wireless communication device for wireless communication, a USB (Universal Serial Bus) connector or an RS232C connector for serial communication, etc. The input device 205 is, for example, a keyboard, a mouse, a touch panel, a button, a microphone, etc. that input data. The output device 206 is, for example, a display, a printer, a speaker, etc. that output data. Each functional unit of the management server 2 described below is realized by the CPU 201 reading a program stored in the storage device 203 into the memory 202 and executing it, and each storage unit of the management server 2 is realized as part of the storage area provided by the memory 202 and the storage device 203.

[0017] 5 is a diagram illustrating an example of the software configuration of the management server 2. The management server 2 includes a learning model storage unit 231, an image acquisition unit 211, an estimation processing unit 212, a light extraction unit 213, and a cornea extraction unit 214.

[0018] <Storage section> The learning model storage unit 231 stores a learning model that has been trained by machine learning on a captured image of a human and the refraction value of the human's eye. As described above, the captured image is an image captured by emitting a flashlight of the user terminal 1 so that the light from the flashlight is irradiated onto the cornea of ​​the human, and then capturing an image of the human with the camera of the user terminal 1, so that the captured image includes light reflected from the cornea. The refraction value of the eye includes at least one of the spherical power, the cylindrical power, and the axis of the cylindrical power.

[0019] The learning model may be created by learning the area of ​​reflected light of a flashlight (hereinafter referred to as a light area) included in the captured image and the refractive value through machine learning. The learning model may also be created by learning the area of ​​the cornea (hereinafter referred to as a corneal area) included in the captured image and the refractive value through machine learning.

[0020] <Functional section> The image acquisition unit 211 acquires an image captured by the user terminal 1 by emitting a flashlight and capturing an image of the user.

[0021] The estimation processing unit 212 provides the acquired captured image to a learning model to estimate the refraction value of the user.

[0022] The light extraction unit 213 can extract a light area in which a flashlight is captured from the captured image. The estimation processing unit 212 can estimate a refraction value by providing the light area to a learning model.

[0023] The cornea extraction unit 214 can extract a corneal region in which the cornea is captured from the captured image. The estimation processing unit 212 can estimate the refractive index by providing the corneal region to a learning model. If the cornea extraction unit 214 cannot extract the corneal region from the captured image (for example, if the eye is closed), the image acquisition unit 211 may transmit a message to the user terminal 1 instructing the user to capture the image again and reacquire the captured image.

[0024] The light extraction unit 213 may extract a light region from the corneal region of the captured image. If the light extraction unit 213 cannot extract a light region from the captured image (for example, when the flashlight is not emitted or the light of the flashlight is not irradiated on the corneal region), the image acquisition unit 211 may transmit a message to the user terminal 1 to instruct the user terminal 1 to capture the image again and reacquire the captured image.

[0025] <Operation> 6 is a diagram for explaining the flow of the learning process. Note that the creation of the learning model may be performed by the user terminal 1 or the management server 2, or may be performed by another computer.

[0026] First, the refraction value of the subject is measured using a general measuring device such as an autorefractometer (S301). Next, the flashlight of the user terminal 1 (smartphone) is turned on and illuminated to the subject's eyes, while the camera of the user terminal 1 captures the subject's image (S302). The measured refraction value and the captured image are stored in a computer in association with each other (S303), and the computer learns the pair of the refraction value and the captured image by machine learning to create a learning model that infers the refraction value from the captured image (S304).

[0027] The learning model created in this manner is stored in the learning model storage unit 231 of the management server 2.

[0028] FIG. 7 is a diagram illustrating the flow of the refraction value prediction process.

[0029] The user terminal 1 emits a flashlight (S321), photographs the user (S322), and transmits the photographed image to the management server 2 (S323). Here, it is assumed that the user terminal 1 is positioned so that light emitted from the flashlight is reflected by the cornea and the reflected light appears in the photographed image.

[0030] The management server 2 receives a photographed image from the user terminal 1 (S324), infers a refractive value by providing the received photographed image to a learning model stored in the learning model storage unit 231 (S325), and outputs the refractive value (for example, transmits the refractive value to the user terminal 1 to display it, or transmits the refractive value to a doctor's terminal to display it) (S326). Note that an image of the eye or cornea extracted from the photographed image may be provided to the learning model.

[0031] The management server 2 may specify the corneal region from the received photographed image, and infer the refraction value by providing the learning model with an extracted image obtained by extracting a portion of the corneal region from the photographed image, or with the photographed image and information specifying the corneal region. The management server 2 may specify the light region from the received photographed image, and infer the refraction value by providing the learning model with an extracted image obtained by extracting a portion of the light region from the photographed image, or with the photographed image and information specifying the light region.

[0032] As described above, according to the refractive power measurement system of the present embodiment, the user's refractive value can be estimated by simply taking a picture of the user by emitting a flashlight using a user terminal 1 such as a smartphone and providing the taken image to the learning model. Therefore, the refractive power of the user can be easily measured.

[0033] Although the present embodiment has been described above, the above embodiment is intended to facilitate understanding of the present invention, and is not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit of the present invention, and equivalents thereof are also included in the present invention.

[0034] For example, in this embodiment, a flashlight is irradiated to the user's eyes, but this is not limiting, and for example, the display of the user terminal 1 may be illuminated and an image of the user may be taken using an in-camera (not shown). In this case, a guide for aligning the position of the user's face (or eyes) may be displayed on the display of the user terminal 1 to fix the position of the face (or eyes). The user terminal 1 can control a part of the display to emit bright white light, so that this light is irradiated to the user's eyes.

[0035] In addition, although the shape of the flashlight is circular in this embodiment, it may be made to emit light in a cross shape, for example. For example, a bright white cross may be displayed on the display, and the user may be photographed using the front camera after the cross is aligned so as to be within the cornea. [Explanation of symbols]

[0036] 1 User terminal 2 Management Server

Claims

1. A learning model storage unit that stores a learning model that is learned by machine learning from an image of an eye taken with a light emitted and irradiated onto a cornea and a refractive value of the eye; an image acquisition unit that acquires an image of the user's eyes by emitting a light from the user terminal; an estimation processing unit that estimates a refractive index of the user by providing the acquired photographed image to the learning model; An information processing system comprising:

2. 2. The information processing system according to claim 1, The refractive value includes at least one of a spherical power, a cylindrical power, and an axis of the cylindrical power; An information processing system comprising:

3. 2. The information processing system according to claim 1, The learning model is a model that learns the light area and the refraction value included in the image, a light extraction unit that extracts a light area in which the light is captured from the captured image; the estimation processing unit estimates the refraction value by providing the light region to the learning model; An information processing system comprising:

4. 2. The information processing system according to claim 1, The learning model is a model that has been trained to learn the area of ​​the cornea and the refractive value included in the image captured so that the light falls on the cornea, a cornea extraction unit that extracts a cornea area from the captured image, the estimation processing unit estimates the refractive value by providing the corneal region to the learning model; An information processing system comprising:

5. A computer that stores a learning model that learns an image of a human eye captured by emitting light and a refractive index of the eye through machine learning, A step of acquiring a captured image of the user's eyes by emitting a light from the user terminal; A step of estimating a refractive index of the user by providing the acquired captured image to the learning model; 2. An information processing method comprising:

6. A computer stores a learning model that learns an image of a human eye taken by emitting light and a refractive value of the eye by machine learning. A step of acquiring a captured image of the user's eyes by emitting a light from the user terminal; A step of estimating a refractive index of the user by providing the acquired captured image to the learning model; A program for executing the above.

Images