Image processing device, image processing method and program

Abstract

A first instruction means 14X of an image processing device 1X instructs a subject to be in a first state in which the subject is looking at a position ahead at a first distance. A first generation means 15X generates first pupil information, which is information relating to the pupils of the subject, on the basis of an image generated by an imaging means when the subject is in the first state. A steadiness assessment means 16X assesses, on the basis of the first pupil information, whether the pupils are steady when the subject is in the first state. If it is assessed that the pupils are steady, a second instruction means 17X instructs the subject to be in a second state in which the subject is looking at a position ahead by a second distance shorter than the first distance. A second generation means 18X generates second pupil information, which is information relating to the pupils of the subject, on the basis of an image generated by the imaging means when the subject is in the second state.

Description

Image processing device, image processing method, and storage medium

[0001] The present disclosure relates to the technical field of an image processing device, an image processing method, and a storage medium that perform processing related to measurement of near vision response using an image.

[0002] There are known systems that measure the pupils of a subject based on a photographed facial image of the subject. For example, Patent Literature 1 discloses a system that detects the area of ​​the subject's pupils based on video signals from two cameras and analyzes the pupil area, pupil diameter, and pupil position.

[0003] International Publication WO2002 / 003853

[0004] Generally, measuring near vision reflexes requires a dedicated device or system. On the other hand, when measuring near vision reflexes on a daily basis for self-care purposes, it is desirable for the subject to be able to easily perform the measurement themselves.

[0005] In view of the above-mentioned problems, one of the objects of the present disclosure is to provide an image processing device, an image processing method, and a storage medium that can suitably generate information regarding a subject's near vision response from an image of the subject.

[0006] One aspect of the image processing device is an image processing device having: a first instruction means for instructing a subject to enter a first state in which the subject looks ahead a first distance; a first generation means for generating first pupil information, which is information about the pupil of the subject, based on an image generated by an imaging means when the subject is in the first state; a constancy determination means for determining whether the pupil of the subject is constancy in the first state based on the first pupil information; a second instruction means for instructing the subject to enter a second state in which the subject looks ahead a second distance shorter than the first distance, if it is determined that the constancy is present; and a second generation means for generating second pupil information, which is information about the pupil, based on an image generated by the imaging means when the subject is in the second state.

[0007] One aspect of the image processing method is an image processing method in which a computer instructs a subject to enter a first state in which the subject looks at something a first distance away, generates first pupil information that is information about the pupil of the subject based on an image generated by an imaging means when the subject is in the first state, determines whether the pupil of the subject is constancy in the first state based on the first pupil information, and if it is determined that the pupil is constancy, instructs the subject to enter a second state in which the subject looks at something a second distance away that is shorter than the first distance, and generates second pupil information that is information about the pupil based on an image generated by the imaging means when the subject is in the second state. Note that the term "computer" includes any electronic device (which may be a processor included in an electronic device) and may be composed of multiple electronic devices.

[0008] One aspect of the storage medium is a storage medium that stores a program that causes a computer to execute the following processes: instructing a subject to enter a first state in which the subject looks ahead a first distance; generating first pupil information, which is information about the subject's pupil, based on an image generated by an imaging means when the subject is in the first state; determining whether the pupil of the subject is constancy in the first state based on the first pupil information; and if it is determined that the pupil is constancy, instructing the subject to enter a second state in which the subject looks ahead a second distance that is shorter than the first distance; and generating second pupil information, which is information about the pupil, based on an image generated by the imaging means when the subject is in the second state.

[0009] It is possible to suitably generate information regarding the subject's near vision response from an image of the subject.

[0010] 1 shows a schematic configuration of a near vision response measurement system according to a first embodiment; FIG. 2 shows a state during measurement of near vision response when the near vision response measurement system is a single terminal device; FIG. 3 shows an example of the hardware configuration of an image processing device common to all embodiments; FIG. 4 shows an example of a functional block of an image processing device relating to near vision response measurement processing in the first embodiment; FIG. 5 shows an example of a camera image capturing a subject's face; FIG. 6 shows an example of a graph illustrating the change over time in pupil size when switching from a distance viewing state to a near viewing state; FIG. 7 shows an example of a graph illustrating the change over time in the degree of convergence recognized when a distance viewing instruction is given again after a near viewing instruction; FIG. 8 shows an example of a flowchart relating to near vision response measurement processing in the first embodiment; FIG. 9 shows a schematic configuration of a near vision response measurement system according to a second embodiment; FIG. 10 is a block diagram of an image processing device according to a third embodiment; FIG. 11 shows an example of a flowchart executed by the image processing device according to the third embodiment.

[0011] Hereinafter, embodiments of an image processing device, an image processing method, and a storage medium will be described with reference to the drawings.

[0012] <First embodiment> (1) System configuration Fig. 1 shows a schematic configuration of a near vision response measurement system 100 according to the first embodiment. The near vision response measurement system 100 is a system that simply measures the near vision response of a subject 6 based on an image of the face of the subject 6 captured by a visible light camera, and mainly includes an image processing device 1, an input device 2, an output device 3, a storage device 4, and a measurement device 5 including a camera (imaging device) 51. The subject 6 simply measures his or her own near vision response using the near vision response measurement system 100, for the purpose of, for example, managing his or her health condition (including self-care).

[0013] The image processing device 1 measures the near reflex of the subject 6 based on facial images of the subject 6 generated by the camera 51 (including a video that is a sequence of a predetermined number of images obtained in time series; the same applies hereinafter), and outputs the measurement results. The image processing device 1 communicates data with the input device 2, output device 3, storage device 4, and measurement device 5 via a communication network or by direct wireless or wired communication.

[0014] The input device 2 is an interface that accepts user input (manual input). The user who inputs information using the input device 2 may be the subject 6 himself / herself, or a person managing or supervising the subject 6. The input device 2 may be, for example, various user input interfaces such as a touch panel, buttons, a keyboard, a mouse, or a voice input device. The input device 2 supplies an input signal generated based on the user input to the image processing device 1.

[0015] The output device 3 outputs predetermined information based on an output signal supplied from the image processing device 1. In this case, the output signal includes at least one of a display signal and an audio signal. The output device 3 displays information based on the display signal supplied from the image processing device 1, and outputs audio information based on the audio signal supplied from the image processing device 1. The output device 3 includes at least one of a display device such as a display or a projector, and an audio output device such as a speaker, for example.

[0016] The storage device 4 is a memory that stores various information necessary for measuring near vision response, etc. The storage device 4 may be an external storage device such as a hard disk connected to or built into the image processing device 1, or may be a storage medium such as a flash memory. The storage device 4 may also be a server device that performs data communication with the image processing device 1. The storage device 4 may also be composed of multiple devices.

[0017] The measurement device 5 is one or more sensors including a camera 51, which is a visible light camera. For example, the measurement device 5 may include an illuminance sensor for detecting changes in the amount of external light in the measurement environment of the subject's 6 near vision response. The measurement device 5 supplies signals measured by each sensor to the image processing device 1. Hereinafter, the image generated by the camera 51 will also be referred to as a "camera image." The camera 51 is an example of an "imaging means."

[0018] 1 is an example, and various modifications may be made to the configuration of the near vision response measurement system 100. For example, the image processing device 1, the input device 2, the output device 3, the storage device 4, and the measurement device 5 may be implemented as a single terminal device such as a smartphone or a tablet terminal.

[0019] Fig. 2 shows a state in which near response is measured when the near response measurement system 100 is a single terminal device (e.g., a smartphone). As shown in Fig. 2, the subject 6 holds the near response measurement system 100, which is a terminal device, and adjusts the orientation of the terminal device so that his or her face is included in the shooting range of the camera 51. The near response measurement system 100 may be fixed to a tripod or the like. Then, in the state shown in Fig. 2, the subject 6 looks at a distance and then at a near object in accordance with instructions (guidance) output by the near response measurement system 100, and causes the near response measurement system 100 to measure the near response.

[0020] (2) Hardware Configuration Fig. 3 shows the hardware configuration of the image processing device 1. The image processing device 1 includes, as hardware, a processor 11, a memory 12, and an interface 13. The processor 11, the memory 12, and the interface 13 are connected via a data bus 90.

[0021] The processor 11 executes programs stored in the memory 12 to function as a controller (arithmetic unit) that performs overall control of the image processing device 1. The processor 11 is, for example, a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0022] The memory 12 is composed of various types of volatile and non-volatile memory, such as RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. The memory 12 also stores programs for executing processes performed by the image processing device 1. Some of the information stored in the memory 12 may be stored in one or more external storage devices capable of communicating with the image processing device 1, or in a storage medium that is detachable from the image processing device 1. The memory 12 may also function as at least a part of the storage device 4.

[0023] The interface 13 is an interface for electrically connecting the image processing device 1 to other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to and from other devices, or may be hardware interfaces for connecting to other devices via cables or the like.

[0024] The hardware configuration of the image processing device 1 is not limited to the configuration shown in Fig. 3. For example, the image processing device 1 may include at least one of an input device 2, an output device 3, a storage device 4, and a measurement device 5.

[0025] (3) Overview of Near Vision Response Measurement Processing Next, a near vision response measurement processing, which is processing related to measuring near vision responses, will be described. In summary, the image processing device 1 determines whether the pupil of the subject 6 has reached a steady state based on camera images generated after instructing the subject 6 to look at a distance. Then, when the pupil of the subject 6 has reached a steady state, the image processing device 1 instructs the subject 6 to look at a near perspective, and calculates an index related to the near vision response based on information related to the subject 6's pupil over time recognized from camera images generated after the instruction. This enables the image processing device 1 to suitably measure near vision responses based on camera images.

[0026] Fig. 4 shows an example of functional blocks of the image processing device 1 related to near vision response measurement processing. Functionally, the processor 11 of the image processing device 1 has a first instruction unit 14, a first pupil information generation unit 15, a constancy determination unit 16, a second instruction unit 17, a second pupil information generation unit 18, and a near vision response output unit 19. Note that in Fig. 4, blocks that exchange data are connected by solid lines, but the combination of blocks that exchange data is not limited to that shown. The same applies to other functional block diagrams described later.

[0027] When the first instruction unit 14 determines that measurement of the near vision response should be started, it instructs the subject 6 to focus on a location at a predetermined distance (also referred to as a "distance vision instruction") so that the subject 6 is in a distance vision state. Specifically, the first instruction unit 14 instructs the subject 6 to focus on a location at a predetermined distance (also referred to as a "first distance"). The "first distance" is a focal distance at which the subject 6 is considered to be in a distance vision state, for example, a distance of 6 meters or more at which the focal distance is essentially infinity. Therefore, for example, the first instruction unit 14 generates an output signal including at least one of a display signal and an audio signal instructing the subject 6 to focus on a location at least 6 meters away, and supplies the output signal to the output device 3 via the interface 13. As a result, the first instruction unit 14 causes the output device 3 to output a distance vision instruction to the subject 6. When the first instruction unit 14 outputs the distance vision instruction, it notifies the first pupil information generation unit 15 that the distance vision instruction has been output.

[0028] Here, a supplementary explanation will be given regarding the determination of whether or not to start measuring the near response. For example, the first instructing unit 14 determines that measurement of the near response should be started when a predetermined user input via the input device 2 instructing the start of measurement of the near response is detected. In another example, the first instructing unit 14 determines that measurement of the near response should be started when a pre-planned timing for measuring the near response arrives. In this case, information regarding the measurement timing is stored in advance in the storage device 4 or the memory 12.

[0029] The first pupil information generation unit 15 generates information about the pupil of the subject 6 based on the camera image acquired from the camera 51 via the interface 13 during the period from when the first instruction unit 14 outputs an instruction until the constancy determination unit 16 determines that the pupil of the subject 6 is constancy. During the above period, the first pupil information generation unit 15 assumes that the subject 6 is looking at a location where the focal length is a first distance (e.g., infinity) (i.e., a state of far-vision, also referred to as the "first state"), and generates information about the pupil of the subject 6 in the state of far-vision. The information about the pupil of the subject 6 is also referred to as "pupil information." In this embodiment, the first pupil information generation unit 15 calculates, as pupil information, at least the pupil size (e.g., pupil diameter) of the subject 6 and the degree of convergence (i.e., a state of closer eyes) (also referred to as "convergence degree"). Specific examples of pupil diameter and convergence degree will be described later. The first pupil information generating section 15 supplies the pupil information (also referred to as “first pupil information”) generated by the first pupil information generating section 15 to the constancy determining section 16 and the near vision response output section 19 .

[0030] After the first instruction unit 14 outputs the far-vision instruction, the constancy determination unit 16 determines whether the pupil of the subject 6 recognized from the camera image has constancy based on the first pupil information. In this case, the constancy determination unit 16 considers the subject 6 to be in a far-vision state after the far-vision instruction is issued by the first instruction unit 14, and determines whether the pupil of the subject 6 recognized in the far-vision state has constancy. If the constancy determination unit 16 determines that the pupil has constancy, it notifies the second instruction unit 17 and the second pupil information generation unit 18 that processing should be started.

[0031] Here, the criteria for determining whether or not constancy exists will be explained in more detail. For example, after a distance vision instruction, the constancy determination unit 16 determines that constancy exists if all of the following conditions (a) to (c) are met continuously for a predetermined time: (a) the focus of the subject 6 does not change; (b) there is no light reaction; and (c) the pupil size is stable. If any of the conditions (a) to (c) is not met continuously for the predetermined time, the constancy determination unit 16 determines that constancy does not exist. The length of the predetermined time is, for example, a default value stored in advance in the storage device 4, the memory 12, or the like.

[0032] First, condition (a) will be described. The constancy determination unit 16 determines whether condition (a) is satisfied based on, for example, the time-series congestion level indicated by the pupil information supplied from the first pupil information generation unit 15 at a predetermined time immediately preceding the condition (a). The constancy determination unit 16 then determines that condition (a) is satisfied if the congestion level has not substantially changed over the predetermined time, and determines that condition (a) is not satisfied if the congestion level has substantially changed over the predetermined time. In this case, for example, the constancy determination unit 16 determines whether the congestion level has substantially changed based on a comparison between a predetermined threshold and a variance, a difference between a maximum value and a minimum value, or other statistical quantity representing variation, of a predetermined number of congestion levels obtained within the predetermined time. The threshold is pre-stored in, for example, the storage device 4 or the memory 12.

[0033] Next, condition (b) will be described. The continuity determination unit 16 determines whether condition (b) is satisfied based on, for example, the time-series light amount (illuminance) measured by the measurement device 5 at a predetermined time immediately preceding the predetermined time. Specifically, the continuity determination unit 16 determines that condition (b) is satisfied if the illuminance measured by the measurement device 5 has not substantially changed over the predetermined time, and determines that condition (b) is not satisfied if the illuminance measured by the measurement device 5 has substantially changed over the predetermined time. In this case, for example, the continuity determination unit 16 determines whether the illuminance has substantially changed based on a comparison of a predetermined threshold value with the variance of a predetermined number of illuminance measurement values ​​obtained within the predetermined time, the difference between the maximum and minimum values, or other statistics representing variation. The threshold value is pre-stored in, for example, the storage device 4 or the memory 12.

[0034] In another example, the constancy determination unit 16 may determine whether or not the condition (b) is satisfied based on the pupil size over time. In this case, the condition (b) is the same as the condition (c) described below.

[0035] Next, condition (c) will be described. The constancy determination unit 16 determines whether condition (c) is satisfied based on, for example, whether there is a substantial change in the time-series pupil size indicated by the pupil information supplied from the first pupil information generation unit 15 at a predetermined time immediately preceding the condition (c). In this case, the constancy determination unit 16 determines whether there is a substantial change in the pupil size based on the results of comparing a predetermined threshold value with the variance of a predetermined number of pupil sizes based on the pupil information obtained within the predetermined time, the difference between the maximum and minimum values, or other statistics representing variation. The threshold value is pre-stored in, for example, the storage device 4 or the memory 12.

[0036] When the constancy determination unit 16 determines that constancy exists, the second instruction unit 17 instructs the subject 6 to focus on a location at a predetermined distance (also referred to as a "near vision instruction") so that the subject 6 is in a near vision state. Specifically, the second instruction unit 17 instructs the subject 6 to focus on a location at a predetermined distance (also referred to as a "second distance"). The "second distance" is a focal distance at which the subject 6 is considered to be in a near vision state, and is a distance shorter than the first distance (e.g., a distance less than 50 cm). For example, in the configuration shown in FIG. 2 (i.e., a configuration in which the subject 6 holds a terminal device that is the near vision reaction measurement system 100), the second instruction unit 17 generates an output signal including at least one of a display signal and an audio signal instructing the subject 6 to focus on a camera 51 provided on the terminal device held by the subject 6, and supplies the output signal to the output device 3 via the interface 13. As a result, the first instruction unit 14 causes the output device 3 to output a near vision instruction to the subject 6. When the first instruction unit 14 outputs a near vision instruction, it notifies the first pupil information generation unit 15 that the near vision instruction has been output.

[0037] The second pupil information generation unit 18 generates time-series pupil information of the subject 6 based on camera images acquired from the camera 51 via the interface 13 during a predetermined time period (e.g., several seconds) from the output of the near vision instruction by the second instruction unit 17. During the above-mentioned period, the first pupil information generation unit 15 assumes that the subject 6 is looking at a location where the focal length is the second distance (i.e., a near vision state, also referred to as the "second state") and generates pupil information including time-series pupil size and convergence of the subject 6 in the near vision state. The second pupil information generation unit 18 supplies the pupil information (also referred to as "second pupil information") generated by the second pupil information generation unit 18 to the near vision response output unit 19.

[0038] The near response output unit 19 calculates an index related to near response (also referred to as a "near response index") based on the first pupil information supplied from the first pupil information generation unit 15 and the second pupil information supplied from the second pupil information generation unit 18, and displays or outputs as sound information on the calculation result of the near response index via the output device 3. In this case, the near response output unit 19 calculates the near response index based on the first pupil information most recently supplied from the first pupil information generation unit 15 (i.e., pupil information in the distance vision state at the time when the constancy determination unit 16 determined that constancy exists) and the time-series second pupil information supplied from the second pupil information generation unit 18. The type of near response index to be calculated varies depending on the application to which the near response measurement system 100 is applied, and specific examples will be described later in sections "(4) Pupil Information and Near Response Index" and "(7) Application."

[0039] Here, the information on the calculation result of the near vision response index that the near vision response output unit 19 outputs to the output device 3 may be information indicating the calculated near vision response index itself, or may be information on the state of the subject 6 estimated based on the near vision response index. An example of the latter will be specifically described in section "(7) Application." The near vision response output unit 19 generates an output signal (i.e., at least one of a display signal and an audio signal) for outputting information on the calculation result of the calculated near vision response index, and supplies the output signal to the output device 3 via the interface 13. As a result, the near vision response output unit 19 causes the output device 3 to output information on the calculation result of the near vision response index.

[0040] The near response output unit 19 may calculate the near response index based on the time-series second pupil information without using the first pupil information. In this case, the second pupil information obtained first in the time series is regarded as pupil information in the far vision state, and the near response index is calculated.

[0041] Furthermore, the image processing device 1 may calculate a predetermined number of near vision response indices by executing the processes of the first instruction unit 14, the first pupil information generation unit 15, the constancy determination unit 16, the second instruction unit 17, the second pupil information generation unit 18, and the near vision response output unit 19 a predetermined number of times. In this case, the near vision response output unit 19 calculates a time average or other representative value of the near vision response indices for the predetermined number of times and outputs the calculation result to the output device 3. This makes it possible to output a highly accurate near vision response index that has been subjected to statistical processing.

[0042] The near vision response output unit 19 may further calculate an index (also referred to as a "distance vision response index") indicating the response of the subject 6 when switching from a near vision state to a distance vision state. In this case, after the second instruction unit 17 issues a near vision instruction, the first instruction unit 14 again issues a distance vision instruction, and the near vision response output unit 19 calculates the distance vision response index based on the time-series first pupil information (and the immediately preceding second pupil information) generated by the first pupil information generation unit 15 after the distance vision instruction. A specific example of this will be described in section "(7) Application."

[0043] The components of the first indicator 14, first pupil information generator 15, constancy determination unit 16, second indicator 17, second pupil information generator 18, and near vision response output unit 19 described in FIG. 4 can be realized, for example, by the processor 11 executing a program. Alternatively, the necessary programs may be recorded on any non-volatile storage medium and installed as needed to realize the components. At least some of these components may not necessarily be realized by software programs, but may also be realized by any combination of hardware, firmware, and software. At least some of these components may also be realized using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the integrated circuit may be used to realize a program consisting of the above components. Furthermore, at least a portion of each component may be configured by an ASSP (Application Specific Standard Product), an ASIC (Application Specific Integrated Circuit), or a quantum processor (quantum computer control chip). In this way, each component may be realized by various hardware. The same applies to other embodiments described below. Furthermore, each of these components may be realized by the cooperation of multiple computers, for example, using cloud computing technology.

[0044] (4) Pupil Information and Near Vision Response Index First, a specific example of pupil information will be described. Fig. 5 shows an example of a camera image of the face of the subject 6.

[0045] The image processing device 1 uses any image recognition technology to recognize the areas of the eyes (both eyes in this case) 9a, iris 9b, and pupil 9c of the subject 6 from a camera image capturing the face of the subject 6. Then, based on each of the recognized areas, the image processing device 1 calculates pupil information indicating the pupil size and degree of convergence, which will be described later.

[0046] For example, the image processing device 1 calculates the pupil size (i.e., pupil diameter) corresponding to the length of the arrow 90. Note that the image processing device 1 may calculate the pupil size for each eye and further generate pupil information indicating the average of the calculated pupil sizes.

[0047] In another example, the image processing device 1 calculates the interpupillary distance corresponding to the length of the arrow 91 as an example of the degree of convergence. In this way, the image processing device 1 can accurately recognize convergence by calculating the degree of convergence based on both eyes. In this case, the camera image is an image capturing at least both eyes of the subject 6, and the image processing device 1 calculates the degree of convergence based on the positions of the pupils 9c of both eyes in the camera image. The image processing device 1 may also calculate pupil information based on a camera image capturing only one eye of the subject. In this case, for example, the image processing device 1 may calculate the relative position of the center of the pupil 9c with respect to both ends of the eye 9a (i.e., the distance corresponding to the lengths of the arrows 93 and 94) as the degree of convergence. In this way, the image processing device 1 may calculate a value indicating the relative position of the pupil 9c in the eye 9a as the degree of convergence.

[0048] Here, the image processing device 1 may perform a normalization process to convert the pupil size, interpupillary distance, and the like from a size based on the number of pixels in the camera image to a size of a predetermined scale (e.g., actual size), and generate pupil information indicating the normalized pupil size and interpupillary distance. In this case, for example, the image processing device 1 may normalize the pupil size and interpupillary distance based on the size of the iris 9b, taking advantage of the characteristics that the size of the iris 9b varies little between individuals and does not change over time. In this case, for example, information indicating the relationship between the size of the iris 9b in the image and the size of the iris 9b after normalization is stored in advance in the storage device 4, memory 12, or the like, and the image processing device 1 performs the above-mentioned normalization by referring to this information. In another example, when a plurality of cameras with different viewpoints are provided as the cameras 51 in the near vision response measurement system 100, the image processing device 1 may recognize the actual pupil size and interpupillary distance by three-dimensionally reconstructing the face of the subject 6 from camera images from the plurality of cameras based on any three-dimensional reconstruction technology such as SfM (Structure From Motion).

[0049] Next, a specific example of a near vision response index will be described. First, a near vision response index obtained from pupil size will be described. Fig. 6 is an example of a graph showing the change over time in pupil size associated with near vision response.

[0050] In this case, after the near vision instruction is given at time "t1," the subject 6 switches from far vision to near vision, causing the pupil size (pupil radius here) of the subject 6 to gradually decrease from the maximum value "Dmax" after a latency period, reaching the minimum value "Dmin" at time "t2." Thereafter, the pupil size expands again and reaches a steady state around time "t3," a predetermined time after time t2.

[0051] In this case, for example, the image processing device 1 calculates, as near vision response indices, the maximum pupil contraction amount corresponding to the length of arrow 95 (i.e., Dmax-Dmin), the maximum pupil contraction rate corresponding to the ratio of size Dmax to size Dmin, the pupil contraction speed corresponding to the slope of line 96, and the re-dilation speed corresponding to the slope of line 97. Note that the slope of line 96 corresponds to the speed of pupil size decrease during the period in which the pupil size decreases, and the slope of line 97 corresponds to the speed of pupil size increase during the period in which the pupil size increases (re-dilation).

[0052] The near vision response index based on pupil size is not limited to the above-described index, but may be any index based on a time series of pupil size measured before and after switching from far vision to near vision. Furthermore, the image processing device 1 may calculate the far vision response index based on pupil size in the same way as the above-described near vision response index.

[0053] Next, the near vision response index related to the degree of convergence will be described. Fig. 7 is an example of a graph showing the time change of the degree of convergence recognized when a near vision instruction is given by the second instruction unit 17 and then a far vision instruction is given by the first instruction unit 14. Here, as an example, the degree of convergence is assumed to be larger as the degree of the eyes becomes higher.

[0054] In the example of Fig. 7, after the near vision instruction is given at time "t11", the subject 6 switches from far vision to near vision, causing the convergence to gradually increase from a minimum level through a latency period and then reach a steady state. Further, at time "t12" when the steady state is reached, the far vision instruction is given, and the subject 6 switches from near vision to far vision, causing the convergence to gradually decrease through a latency period and reach a steady state around time "t13".

[0055] In this case, for example, the image processing device 1 calculates, as a near vision response index, the difference between the minimum value of the convergence corresponding to the distance vision state and the maximum value of the convergence corresponding to the distance vision state (corresponding to the length of the arrow 95a) and the rate of increase of the convergence (corresponding to the slope of the arrow 97a), as in the example of FIG. 6 . Furthermore, for example, the image processing device 1 calculates, as a far vision response index, the difference between the maximum value of the convergence corresponding to the distance vision state and the minimum value of the convergence corresponding to the distance vision state (corresponding to the length of the arrow 98a) and the rate of decrease of the convergence during the period of decrease of the convergence accompanying switching from near vision to distance vision (corresponding to the slope of the arrow 96a). Here, the difference between the maximum values ​​of the convergence is the degree of change in the convergence, and is hereinafter also referred to as the "amount of convergence." Furthermore, the above-mentioned rate of increase and rate of decrease are hereinafter also referred to as the "rate of change of the convergence." The amount of convergence and the rate of change of the convergence are examples of "indices related to the change in the degree of convergence."

[0056] The near vision response index based on the degree of convergence is not limited to the above-mentioned index, but may be any index based on the time series of the degree of convergence measured in the period before and after switching from far vision to near vision.Similarly, the far vision response index based on the degree of convergence is not limited to the above-mentioned index, but may be any index based on the time series of the degree of convergence measured in the period before and after switching from near vision to far vision.

[0057] (5) Processing Flow FIG. 8 is an example of a flowchart relating to the near vision response measurement processing executed by the image processing device 1.

[0058] First, the image processing device 1 outputs a distance vision instruction to the subject 6 via the output device 3 (step S11). Then, the image processing device 1 generates pupil information (i.e., first pupil information) of the subject 6 based on each of the time-series camera images generated by the camera 51 after step S11 (step S12).

[0059] The image processing device 1 then determines whether or not the pupil of the subject 6 is constancy based on the pupil information etc. obtained in step S12 (step S13). In this case, the image processing device 1 determines whether or not the pupil is constancy based on a predetermined number of pieces of pupil information generated in the previous predetermined period in step S12. If the image processing device 1 determines that the pupil of the subject 6 is constancy (step S13; Yes), the process proceeds to step S14. On the other hand, if the image processing device 1 determines that the pupil of the subject 6 is not constancy (step S13; No), the image processing device 1 continues to generate pupil information based on the latest camera image generated by the camera 51 in step S12.

[0060] Next, after determining that the pupil of the subject 6 is stationary, the image processing device 1 outputs a near vision instruction to the subject 6 via the output device 3 (step S14). The image processing device 1 then generates pupil information (i.e., second pupil information) of the subject 6 based on each of the time-series camera images generated by the camera 51 after step S14 (step S15). The image processing device 1 then calculates a near vision response index based on the pupil information (step S16). In this case, the image processing device 1 calculates one or more types of near vision response indexes based on the time-series second pupil information obtained in step S15 (and the first pupil information obtained in step S12). The image processing device 1 may also calculate a time average (or other representative value) of each type of near vision response index based on multiple samples of near vision response indexes obtained by repeating steps S11 to S16 multiple times.

[0061] Then, the image processing device 1 outputs information related to the calculation result of the near vision response index via the output device 3 (step S17). In this case, the image processing device 1 may determine the state of the subject 6, such as the presence or absence of aging phenomena, fatigue, or nervous system disease, from the calculated near vision response index according to the application to be applied, and output the determination result via the output device 3. Examples of applications will be described later.

[0062] (6) Modifications Modifications of the above-described embodiment will now be described. The following modifications may be applied to the above-described embodiment in any combination.

[0063] (Variation 1) When instructing far vision and near vision, the image processing device 1 may output guidance so that the distance (shooting distance) between the subject 6 and the camera 51 is a predetermined distance based on the size of the iris of the subject 6 recognized from the camera image.

[0064] In this case, for example, in step S11, the image processing device 1 outputs a distance vision instruction and recognizes the iris size of the subject 6 from the camera image. The image processing device 1 then determines whether the recognized iris size falls within an appropriate size range. The aforementioned appropriate size range is an iris size range corresponding to a shooting distance range that is within a distance range preferable for near vision response measurement processing, and is a range pre-stored in, for example, the storage device 4 or the memory 12. If the recognized iris size falls outside the appropriate size range, the image processing device 1 outputs to the output device 3 information instructing the subject 6 to adjust the shooting distance. In this case, if the recognized iris size is smaller than the appropriate size range, the image processing device 1 instructs the subject 6 to shorten the shooting distance, and if the recognized iris size is larger than the appropriate size range, the image processing device 1 instructs the subject 6 to increase the shooting distance. Similarly, in step S14 and other steps, the image processing device 1 may recognize the iris size of the subject 6 from the camera image and output guidance regarding the shooting distance based on the determination result of whether the recognized iris size falls within the appropriate size range.

[0065] According to this modification, the image processing device 1 can suitably adjust the shooting distance to a distance suitable for measuring near vision response.

[0066] (Modification 2) Acquiring the second pupil information in time series is not essential, and the image processing device 1 may acquire the second pupil information at least at one time point in a near vision state.

[0067] In this case, for example, the near response output unit 19 calculates a near response index based on second pupil information based on a camera image generated at a predetermined time after the second instruction unit 17 issues a far-vision instruction and first pupil information generated last before the second instruction unit 17 issues a far-vision instruction. The predetermined time may be, for example, a time between time t1 and time t2 in FIG. 6 (i.e., a time during the period when the pupil size is decreasing) or a time around steady time t3 (i.e., a time when the pupil state reaches a steady state). In the former example, the near response output unit 19 can calculate the difference between the pupil size indicated by the first pupil information and the pupil size indicated by the second pupil information as the maximum pupil contraction rate. In the latter example, the near response output unit 19 can calculate the difference between the pupil size indicated by the first pupil information and the pupil size indicated by the second pupil information as the difference between the pupil size for near vision and that for far vision in the steady state.

[0068] (7) Applications Next, examples of applications of the near vision response measurement system 100 will be described. In the following applications, the near vision response measurement system 100 estimates the state (particularly the state related to health) of the subject 6 based on the calculated near vision response index, and outputs the estimated state of the subject 6. Here, as specific examples of applications of the near vision response measurement system 100, an application relating to quantitative understanding of functional decline due to aging, an application relating to quantitative understanding of eye strain, and an application relating to detection of binocular vision abnormalities will be described.

[0069] (7-1) Quantitative Assessment of Age-Related Functional Decline In an application related to quantitative assessment of age-related functional decline, for example, the image processing device 1 calculates, as near vision response indices, the pupil size in a distance vision state (i.e., the maximum pupil size Dmax), the pupil size in a near vision state (i.e., the minimum pupil size Dmin or the pupil size after re-dilation), the pupil contraction speed, the amount of convergence, and the rate of change in the convergence. The image processing device 1 then compares the calculated values ​​of various near vision response indices with reference values ​​of the near vision response indices to quantitatively estimate the degree of age-related functional decline of the subject 6. The image processing device 1 then outputs the estimation results to the output device 3. The reference values ​​may be general reference values ​​of various near vision response indices for the age of the subject 6, or may be past calculated values ​​of the near vision response indices of the subject 6. The reference values ​​may also be thresholds for determining the presence or level of age-related functional decline.

[0070] Here, the image processing device 1 may use a model for estimating the degree of functional decline due to aging of the subject 6, and output information output by the model to the output device 3. The above-mentioned model is a machine learning model such as an equation, a lookup table, or a neural network, and outputs an estimation result regarding the degree of functional decline due to aging (e.g., an estimated age of the subject 6) when, for example, calculated values ​​of various near vision response indices (or differences between the calculated values ​​and reference values, etc.) are input. Parameters of the above-mentioned model, etc., are stored in advance in the storage device 4, the memory 12, etc. Furthermore, the image processing device 1 may display a graph or the like that allows comparison between the calculated values ​​of various near vision response indices and the corresponding reference values.

[0071] With such an application, the subject 6 can easily measure his / her near vision response by himself / herself using a smartphone or the like, and quantitatively grasp the functional decline due to aging. The image processing device 1 can also suggest preventive activities for functional decline due to aging and visualize the preventive effects of functional decline due to aging, thereby increasing the user (subject 6)'s awareness of continuing to take preventive activities for functional decline due to aging.

[0072] (7-2) Quantitative Assessment of Eye Strain In an application related to quantitative assessment of eye strain, the image processing device 1 calculates, for example, the maximum pupil contraction rate, pupil contraction velocity, re-dilation velocity, and convergence amount as near vision response indices. The image processing device 1 then compares the calculated values ​​of various near vision response indices with reference values ​​of the near vision response indices to quantitatively estimate the degree of eye strain of the subject 6. The image processing device 1 then outputs the estimation result to the output device 3. The reference values ​​may be general reference values ​​of various near vision response indices for the age of the subject 6, or may be previously calculated values ​​of the near vision response indices of the subject 6. The reference values ​​may also be thresholds for determining the presence or absence or level of eye strain of the subject 6.

[0073] Here, the image processing device 1 may use a model for estimating the degree of eye strain of the subject 6 and cause the output device 3 to output information output by the model. The above-mentioned model is a machine learning model such as an equation, a lookup table, or a neural network, and outputs an estimation result regarding the degree of eye strain of the subject 6 when, for example, calculated values ​​of various near vision response indices (or differences between the calculated values ​​and reference values, etc.) are input. Parameters of the above-mentioned model, etc., are stored in advance in the storage device 4, memory 12, etc. Furthermore, the image processing device 1 may display, as information indicating the degree of eye strain of the subject 6, a graph or the like that compares the calculated values ​​of various near vision response indices with the corresponding reference values.

[0074] According to such an application, the subject 6 can easily measure his / her near vision response by himself / herself using a smartphone or the like that he / she owns, and quantitatively grasp his / her eye strain.

[0075] (7-3) Detection of Binocular Vision Abnormalities Next, an application for detecting binocular vision abnormalities (including esotropia caused by the use of a smartphone) will be described.

[0076] In this application, the image processing device 1 calculates, for example, the maximum pupil contraction rate, pupil contraction velocity, amount of convergence, and rate of change of the convergence rate as near vision response indices. Furthermore, in this application, the image processing device 1 calculates a distance vision response index in addition to the near vision response index. Therefore, after issuing a near vision instruction using the second instruction unit 17, the image processing device 1 again issues a distance vision instruction using the first instruction unit 14, and calculates the distance vision response index based on the time-series first pupil information (and the immediately preceding second pupil information) generated by the first pupil information generation unit 15 after the distance vision instruction. As a result, the image processing device 1 calculates the pupil dilation rate, pupil dilation velocity, amount of convergence, rate of change of the convergence rate, and other indices associated with the subject 6's response to switching from a near vision state to a distance vision state. The "pupil dilation ratio" is the ratio of the pupil size in the near vision state to the pupil size in the subsequent far vision state, and the "pupil dilation speed" is the speed at which the pupil size dilates during the period when the pupil size dilates due to switching from the near vision state to the far vision state.

[0077] The image processing device 1 then compares the calculated values ​​of the various near vision response indices and distance vision response indices with reference values ​​of the various near vision response indices and distance vision response indices to estimate binocular vision abnormalities of the subject 6. The image processing device 1 then outputs the estimation results to the output device 3. The reference values ​​may be general reference values ​​of the various near vision response indices and distance vision response indices for the age of the subject 6, or may be previously calculated values ​​of the near vision response indices and distance vision response indices of the subject 6. The reference values ​​may also be thresholds for determining the presence or absence or level of binocular vision abnormalities of the subject 6.

[0078] In addition, if the subject 6 experiences either a state in which the pupil does not constrict even when convergence occurs in the near vision response, or a state in which the degree of convergence does not return to the original degree of convergence in the far vision response when switching from the near vision state to the far vision state, there is a high possibility that a binocular vision abnormality has occurred. Therefore, the above-mentioned threshold value should be set to be a reference value for determining whether or not the subject 6 experiences either of the above-mentioned states.

[0079] With this application, the subject 6 can easily measure his / her near vision response by himself / herself using his / her own smartphone or the like, and understand the results of the binocular vision abnormality assessment. Furthermore, the image processing device 1 can also suggest preventive activities according to the binocular vision abnormality assessment results and visualize the preventive effects, thereby increasing the subject 6's awareness of continuing preventive activities for binocular vision abnormalities. This application can be used not only for self-checks by the subject 6, but also for checks by parents if the subject 6 is a minor.

[0080] <Second embodiment> Fig. 9 shows a schematic configuration of a near vision response measurement system 100A in the second embodiment. The near vision response measurement system 100A according to the second embodiment has an image processing device 1A functioning as a server and a terminal device 8 used by a subject and functioning as a client. The image processing device 1A and the terminal device 8 communicate data via a network 99. Hereinafter, the same components as those in the first embodiment will be appropriately designated by the same reference numerals, and their description will be omitted.

[0081] The terminal device 8 is a terminal used by a user who will be the subject, and has input, display, communication, and imaging functions, and functions as the input device 2, output device 3, and measurement device 5 including camera 51 shown in Fig. 1. The terminal device 8 may be, for example, a personal computer, a tablet terminal such as a smartphone, or a PDA (Personal Digital Assistant). The terminal device 8 transmits the facial image of the subject output by the camera 51 to the image processing device 1A via the network 99.

[0082] The image processing device 1A has the same hardware configuration as the image processing device 1 shown in Fig. 2, and the processor 11 of the image processing device 1A has the functional blocks shown in Fig. 4 described in the first embodiment. The image processing device 1A receives camera images from the terminal device 8 via the network 99 and executes near vision reflex measurement processing of the subject. Furthermore, the image processing device 1A transmits an output signal for outputting the processing results to the terminal device 8 via the network 99 based on a display request from the terminal device 8.

[0083] In this way, the image processing device 1A in the second embodiment performs processing related to measuring the near vision response of the subject who is the user of the terminal device 8, and can present the measurement results of the near vision response to the subject via the terminal device 8 in an appropriate manner.

[0084] 10 is a block diagram of an image processing device 1X according to a third embodiment. The image processing device 1X mainly includes a first instruction unit 14X, a first generation unit 15X, a continuity determination unit 16X, a second instruction unit 17X, and a second generation unit 18X. Note that the image processing device 1X may be configured using multiple devices.

[0085] The first instructing unit 14X instructs the subject to be in a first state in which the subject views an object a first distance ahead. The first instructing unit 14X can be, for example, the first instructing unit 14 in the first or second embodiment.

[0086] The first generating unit 15X generates first pupil information, which is information about the pupil of the subject, based on the image generated by the imaging unit when the subject is in the first state. The first generating unit 15X can be, for example, the first pupil information generating unit 15 in the first or second embodiment.

[0087] The constancy determination means 16X determines whether or not the pupil of the subject is in the first state based on the first pupil information. The constancy determination means 16X can be, for example, the constancy determination unit 16 in the first or second embodiment.

[0088] When it is determined that steadiness exists, the second instructing unit 17X instructs the subject to enter a second state in which the subject looks ahead at a second distance that is shorter than the first distance. The second instructing unit 17X can be, for example, the second indicator 17 in the first or second embodiment.

[0089] The second generating unit 18X generates second pupil information, which is information about the pupil of the subject, based on the image generated by the imaging unit when the subject is in the second state. The second generating unit 18X can be, for example, the second pupil information generating unit 18 in the first or second embodiment.

[0090] 11 is an example of a flowchart executed by the image processing device 1X in the third embodiment. First, the first instructing unit 14X instructs the subject to enter a first state in which the subject gazes at a first distance (step S21). The first generating unit 15X generates first pupil information, which is information about the subject's pupil, based on an image generated by the imaging unit when the subject is in the first state (step S22). The constancy determining unit 16X determines whether the subject's pupil is constancy in the first state based on the first pupil information (step S23). If the second instructing unit 17X determines that constancy exists, it instructs the subject to enter a second state in which the subject gazes at a second distance, which is shorter than the first distance (step S24). The second generating unit 18X generates second pupil information, which is information about the subject's pupil, based on an image generated by the imaging unit when the subject is in the second state (step S25).

[0091] According to the third embodiment, the image processing device 1X can suitably acquire pupil information associated with the near vision response of the subject.

[0092] In the above-described embodiments, the program can be stored using various types of non-transitory computer-readable media and supplied to a computer processor or the like. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can be supplied to a computer via wired communication paths such as electric wires and optical fibers, or via wireless communication paths.

[0093] In addition, some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes.

[0094] [Supplementary Note 1] An image processing device comprising: a first instructing means for instructing a subject to enter a first state in which the subject looks at something a first distance away, a first generating means for generating first pupil information that is information about the pupil of the subject based on an image generated by an imaging means when the subject is in the first state, a constancy determining means for determining whether the pupil of the subject is constancy in the first state based on the first pupil information, a second instructing means for instructing the subject to enter a second state in which the subject looks at something a second distance away that is shorter than the first distance when the constancy is determined to be present, and a second generating means for generating second pupil information that is information about the pupil based on an image generated by the imaging means when the subject is in the second state. [Supplementary Note 2] The image processing device according to Supplementary Note 1 further comprises a near vision response calculating means for calculating an index related to the near vision response of the subject based on the second pupil information. [Supplementary Note 3] The image processing device according to Supplementary Note 2, wherein the near response calculation means calculates the index based on the second pupil information in time series generated after an instruction to enter the second state is given. [Supplementary Note 4] The image processing device according to Supplementary Note 1, wherein the first pupil information and the second pupil information include information on the degree of convergence and the size of the pupil. [Supplementary Note 5] The image processing device according to Supplementary Note 4, wherein the image is an image obtained by photographing at least both eyes of the subject, and the first generation means and the second generation means calculate the degree of convergence based on positions of the pupils of both eyes in the image. [Supplementary Note 6] The image processing device according to Supplementary Note 1, wherein the constancy determination means determines whether the pupil is constancy based on whether there is a change in focus, whether there is a pupillary light reflex, and whether there is a change in the size of the pupil. [Appendix 7] The image processing device described in Appendix 2, wherein the first instructing means, after generating the second pupil information, re-instructs the subject to enter the first state, and the near vision response calculating means calculates an index related to the response when the subject switches from the second state to the first state based on the second pupil information and the first pupil information generated after the re-instruction.[Supplementary Note 8] The image processing device according to Supplementary Note 2, wherein the near response calculation means calculates, as the index, at least one of the maximum value of the pupil size, the minimum value of the pupil size, the maximum contraction rate of the pupil, the contraction speed of the pupil, the re-dilation speed of the pupil, or an index relating to a change in the degree of convergence. [Supplementary Note 9] The image processing device according to Supplementary Note 2, wherein the near response calculation means outputs, via an output device, information relating to the condition of the subject estimated based on the index. [Supplementary Note 10] The image processing device according to Supplementary Note 9, wherein the near response calculation means outputs, as the information relating to the condition, the degree of functional decline due to aging of the subject estimated based on the index. [Supplementary Note 11] The image processing device according to Supplementary Note 9, wherein the near response calculation means outputs, as the information relating to the condition, the degree of eyestrain of the subject estimated based on the index. [Supplementary Note 12] The image processing device according to Supplementary Note 9, wherein the near response calculation means outputs, as the information relating to the condition, information relating to binocular vision abnormality of the subject estimated based on the index. [Supplementary Note 13] An image processing method, in which a computer instructs a subject to enter a first state in which the subject looks ahead a first distance, generates first pupil information that is information about the pupil of the subject based on an image generated by an imaging means when the subject is in the first state, determines whether the pupil of the subject is constancy in the first state based on the first pupil information, and if it is determined that the pupil is constancy, instructs the subject to enter a second state in which the subject looks ahead a second distance that is shorter than the first distance, and generates second pupil information that is information about the pupil based on an image generated by the imaging means when the subject is in the second state.[Supplementary Note 14] A storage medium storing a program that causes a computer to execute the following processes: instructing a subject to enter a first state in which the subject looks ahead at a first distance; generating first pupil information that is information about the pupil of the subject based on an image generated by an imaging means when the subject is in the first state; determining whether the pupil of the subject is constancy in the first state based on the first pupil information; and if it is determined that the pupil is constancy, instructing the subject to enter a second state in which the subject looks ahead at a second distance that is shorter than the first distance; and generating second pupil information that is information about the pupil based on an image generated by the imaging means when the subject is in the second state.

[0095] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications within the scope of the present invention that would be understood by those skilled in the art can be made to the configuration and details of the present invention. In other words, the present invention naturally includes various modifications and alterations that would be possible for those skilled in the art based on the entire disclosure, including the claims, and the technical ideas. Furthermore, the disclosures of the above-cited patent and non-patent documents are incorporated herein by reference.

[0096] REFERENCE SIGNS LIST 1, 1A, 1X Image processing device 2 Input device 3 Output device 4 Storage device 5 Measurement device 8 Terminal device 11 Processor 12 Memory 13 Interface 51 Camera 90 Data bus 99 Network 100, 100A Near vision response measurement system

Claims

1. a first instruction means for instructing the subject to assume a first state in which the subject visually recognizes an object a first distance ahead; a first generating means for generating first pupil information, which is information about a pupil of the subject, based on an image generated by an imaging means when the subject is in the first state; a constancy determination means for determining whether or not the pupil of the subject is constancy in the first state based on the first pupil information; a second instruction means for instructing the subject to enter a second state in which the subject looks ahead at a second distance that is shorter than the first distance when it is determined that the steadiness exists; a second generating means for generating second pupil information, which is information about the pupil, based on the image generated by the imaging means when the subject is in a second state; An image processing device having:

2. The image processing apparatus according to claim 1 , further comprising a near vision response calculation unit that calculates an index relating to the near vision response of the subject based on the second pupil information.

3. The image processing device according to claim 2 , wherein the near vision response calculation means calculates the index based on the second pupil information in time series that is generated after an instruction to enter the second state is given.

4. The image processing device according to claim 1 , wherein the first pupil information and the second pupil information include information relating to a degree of convergence and a size of the pupil.

5. the image is an image of at least both eyes of the subject, The image processing device according to claim 4 , wherein the first generating means and the second generating means calculate the degree of convergence based on positions of the pupils of the eyes in the image.

6. The image processing device according to claim 1 , wherein the constancy determining means determines whether the pupil is constancy based on whether there is a change in focus, whether there is a light reflex, and whether there is a change in the size of the pupil.

7. the first instruction means, after generating the second pupil information, instructs the subject again to be in the first state; 3. The image processing device according to claim 2, wherein the near vision response calculation means calculates an index relating to a response when the subject switches from the second state to the first state based on the first pupil information generated after the re-instruction.

8. 3. The image processing device according to claim 2, wherein the near response calculation means calculates, as the index, at least one of a maximum value of the pupil size, a minimum value of the pupil size, a maximum pupil contraction rate, a pupil contraction speed, a pupil re-dilation speed, or an index related to a change in a degree of convergence.

9. The computer instructing the subject to assume a first state in which the subject looks a first distance ahead; generating first pupil information, which is information about the pupil of the subject, based on an image generated by an imaging means when the subject is in the first state; determining whether or not the pupil of the subject is in the first state based on the first pupil information; When it is determined that the steadiness exists, instruct the subject to enter a second state in which the subject looks ahead at a second distance that is shorter than the first distance; generating second pupil information, which is information about the pupil, based on the image generated by the imaging means when the subject is in a second state; Image processing methods.

10. instructing the subject to assume a first state in which the subject looks a first distance ahead; generating first pupil information, which is information about the pupil of the subject, based on an image generated by an imaging means when the subject is in the first state; determining whether or not the pupil of the subject is in the first state based on the first pupil information; When it is determined that the steadiness exists, instruct the subject to enter a second state in which the subject looks ahead at a second distance that is shorter than the first distance; A program that causes a computer to execute a process of generating second pupil information, which is information about the pupil, based on an image generated by the imaging means when the subject is in a second state.