Program, display device, and information processing system

Abstract

To provide a program, a display device, and an information processing system capable of improving reproducibility when digitizing a positional relationship of components of a measurement object.SOLUTION: A program causes a computer including an imaging unit and an output unit that causes a display unit to display an image of a measurement site of a subject captured by the imaging unit to execute a step of determining a two dimensional coordinate having a reference line passing through a position of the image of the measurement site as an axis in an image including the image of the measurement site of the subject and an image of a marker having a predetermined size, and detecting a distance in an axial direction between two points to be measured in the measurement site from the image including the image of the measurement site and the image of the marker using the two dimensional coordinate.SELECTED DRAWING: Figure 3B

Description

[Technical Field]

[0001] The present invention relates to a program, a display device, and an information processing system. [Background technology]

[0002] External conditions such as appearance may be quantified. For example, measuring the positions of components related to external conditions such as appearance may be used in plastic surgery, cosmetic surgery, ophthalmology, aesthetic treatments, and other medical procedures. In plastic surgery and other fields, there is a need to quantify the positions of the components of the target area when formulating a treatment plan and determine the treatment strategy. In cosmetic surgery and other fields, there is a need to quantify the positions and sizes of facial features when formulating a treatment plan and measure the balance of each facial feature to determine the treatment strategy. In ophthalmology, ptosis is known to be a condition in which external conditions affect the field of vision, and when ptosis occurs, the eyelids may droop, obstructing the field of vision and affecting the patient's quality of life. While ptosis caused by aging or other factors is not life-threatening, the severity of the symptoms can affect the patient's quality of life. When ptosis symptoms progress, surgical treatment may be selected. When performing the above-mentioned surgical procedures, it is required to quantify the positional relationships of components before, during, and after the procedure. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6616541 Summary of the Invention [Problem to be solved by the invention]

[0004] The above-mentioned Patent Document 1 relates to a technology for measuring the color intensity of makeup application using a terminal device, and even with the technology of the above-mentioned Patent Document 1, it was not easy to improve the reproducibility when quantifying the positional relationship of components. When measuring using a ruler for the above-mentioned quantification, the measurement work becomes complicated as the number of measurements and measurement locations increase, and errors due to measurement variations become a factor that reduces reproducibility.

[0005] The present invention has been made to solve the above-mentioned problems, and aims to provide a program, a display device, and an information processing system that can improve reproducibility when quantifying the positional relationship of components to be measured. [Means for solving the problem]

[0006] A program according to one embodiment of the present invention is a program for causing a computer having an imaging unit and an output unit that displays on a display unit an image of the measurement site of a subject imaged by the imaging unit to execute the steps of: defining two-dimensional coordinates on an image including an image of the measurement site of the subject and an image of a marker of a predetermined size, with a reference line passing through the position of the image of the measurement site as its axis; and using the two-dimensional coordinates to detect the axial distance between two points of the subject at the measurement site from the image including the image of the measurement site and the image of the marker.

[0007] Furthermore, a program according to one embodiment of the present invention includes a method in which the measurement site of the subject includes the left and right eyes of the subject's face, the reference line in the image is positioned near the inner corner of the eye, and the reference line is used as an axis of a two-dimensional coordinate system to detect the axial distance between two points of the measurement object from an image including an image of the subject's face.

[0008] In addition, a program according to one embodiment of the present invention includes a step of generating scale information for an image including an image of the subject's face from the size of the image of the marker reflected in the image, and detecting the axial distance between two points on the measurement object using the generated scale information.

[0009] Moreover, the program according to one aspect of the present invention includes a step of deriving the distance from the reference line to the image of the measurement object as the distance between the reference line and a line segment parallel to the reference line.

[0010] In addition, a program according to one aspect of the present invention includes steps of: placing the reference line on an image of the subject's eye; extracting the position of the subject's inner corner of the eye; and displaying an indicator at the position of the subject's inner corner of the eye.

[0011] In addition, in a program according to one aspect of the present invention, the measurement targets when the eyes are open include any of the subject's pupil center (pupil center line), upper eyelid margin (edge ​​of the upper eyelid), double eyelid line (double eyelid line), or lower eyebrow edge (line below the eyebrow).

[0012] In addition, in a program according to one aspect of the present invention, the measurement target when the eyes are closed includes any one of the subject's upper eyelid margin (edge ​​of the upper eyelid), double eyelid line (double eyelid line), or lower eyebrow margin (line below the eyebrow).

[0013] In addition, in a program according to one aspect of the present invention, the measurement targets when the subject's eyes are open and their gaze is directed upward with their face facing forward include any of the subject's pupil center (pupil center line), upper eyelid margin (edge ​​of the upper eyelid), or lower eyebrow margin (line below the eyebrow).

[0014] Furthermore, a program according to one aspect of the present invention is a program for executing a step of identifying from the image any of the subject's pupil center (pupil center line), upper eyelid margin (edge ​​of the upper eyelid), double eyelid line (double eyelid line), or lower eyebrow margin (line below the eyebrow).

[0015] Furthermore, a program according to one embodiment of the present invention includes detecting the axial distance between two points of the measurement object using index values ​​on the facial center line, with the reference line and a facial center line perpendicular to the reference line as the axes of the two-dimensional coordinate system.

[0016] Furthermore, a program according to one aspect of the present invention is a program for executing the steps of generating a reference line to be used as an axis of a two-dimensional coordinate system, displaying the reference line superimposed on an image of the measurement site of the subject, and capturing an image of the measurement site of the subject at a predetermined timing within the period during which the reference line is displayed to obtain image data including an image of the measurement site of the subject and an image of a marker of a predetermined size.

[0017] In addition, a program according to one aspect of the present invention is a program for executing a step of capturing an image of a measurement site including the subject's eyes, recording data indicating the degree of the subject's ptosis symptoms, and making the data on the degree of the subject's ptosis symptoms possible to view or compare with historical data.

[0018] A display device according to one aspect of the present invention comprises an imaging unit, a display unit that displays an image of a measurement site of a subject captured by the imaging unit, a control unit that controls the imaging by the imaging unit and the display by the display unit, and an arithmetic processing unit that performs predetermined calculations. The control unit generates a reference line to be used as an axis of a two-dimensional coordinate system and displays the reference line superimposed on the image of the measurement site of the subject. The control unit images the measurement site of the subject at a predetermined timing within the period during which the reference line is displayed to obtain image data including an image of the measurement site of the subject and an image of a marker of a predetermined size. The arithmetic processing unit uses the reference line, which passes through the position of the image of the measurement site in the image including the image of the measurement site of the subject and the image of the marker of the predetermined size, as the axis of the two-dimensional coordinate system, and detects the axial distance between two points on the measurement target at the measurement site from the image including the image of the measurement site and the image of the marker.

[0019] An information processing system according to one embodiment of the present invention is an information processing system that uses a computer having an imaging unit and a display unit that displays an image of a measurement site of a subject imaged by the imaging unit, and causes the computer to determine two-dimensional coordinates on an image including an image of the measurement site of the subject and an image of a marker of a predetermined size, with a reference line passing through the position of the image of the measurement site as its axis, and to use the two-dimensional coordinates to detect the axial distance between two points of the subject at the measurement site from the image including the image of the measurement site and the image of the marker. [Effects of the Invention]

[0020] According to the program, display device, and information processing system of the present invention, it is possible to improve the reproducibility when quantifying the positional relationship of components of a measurement target. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a schematic configuration diagram of an information processing system according to an embodiment of the present invention. [Figure 2A] FIG. 1 is a diagram illustrating an information processing system according to an embodiment. [Figure 2B] FIG. 10 is a diagram for explaining acquisition of a face image according to the embodiment. [Figure 3A] FIG. 4 is a diagram for explaining measurement points according to the embodiment. [Figure 3B] FIG. 4 is a diagram for explaining measurement points according to the embodiment. [Figure 3C] FIG. 4 is a diagram for explaining measurement points according to the embodiment. [Figure 3D] FIG. 4 is a diagram for explaining measurement points according to the embodiment. [Figure 3E] FIG. 4 is a diagram for explaining measurement points according to the embodiment. [Figure 4] FIG. 2 is a diagram for explaining a table for storing position information for defining two-dimensional coordinates according to the embodiment. [Figure 5]FIG. 2 is a diagram for explaining a table for storing position information of a measurement target according to the embodiment. [Figure 6] FIG. 10 is a diagram for explaining a table for storing information on evaluation results related to ptosis according to the embodiment. [Figure 7A] 1 is a configuration diagram of a terminal device and a history data processing device according to an embodiment. [Figure 7B] FIG. 2 is a configuration diagram of an administrator terminal according to the embodiment. [Figure 8] FIG. 10 is a diagram for explaining a detection process according to the embodiment. [Figure 9] 10 is a flowchart of a detection process according to the embodiment. [Figure 10] 10A and 10B are explanatory diagrams relating to a process of determining a feature amount of a detection target according to a first method of the embodiment. [Figure 11] 10A and 10B are diagrams for explaining processing for one eye in an open eye state according to an embodiment. [Figure 12] 10A and 10B are explanatory diagrams relating to a process of determining a feature amount of a detection target according to a second method of the embodiment. [Figure 13] FIG. 10 is a diagram for explaining a discrimination process according to the embodiment. [Figure 14] FIG. 10 is a schematic configuration diagram of an information processing system according to a third embodiment. [Figure 15] FIG. 10 is a diagram illustrating an information processing system according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0022] An information processing system according to an embodiment will be described below with reference to the drawings. The following description is for explaining an embodiment of this disclosure, and the disclosure is not limited to the following embodiment. For clarity of explanation, the following description has been omitted or simplified as appropriate. Furthermore, each element of the following embodiment can be easily modified, added, or converted within the scope of this disclosure. In the embodiment, the information processing system can be applied to medical treatments such as plastic surgery, cosmetic surgery (cosmetic surgery), ophthalmology, etc. There is no limitation on the application, and the information processing system may be applied during aesthetic treatment.

[0023] (First embodiment) An information processing system 1 according to an embodiment will be described with reference to FIGS. Fig. 1 is a schematic configuration diagram of an information processing system 1 according to an embodiment. Fig. 2A is a diagram for explaining the information processing system according to an embodiment. Fig. 2B is a diagram for explaining acquisition of a face image according to an embodiment. The information processing system 1 allows a program registered in the management device 300 to be used on the terminal device 100 of the user US1 based on a usage request from the terminal device 100. The program is distributed via the network 5, for example.

[0024] The information processing system 1 includes, for example, a terminal device 100, an administrator terminal 200, a management device 300, and a history data processing device 500. The terminal device 100, the administrator terminal 200, the management device 300, and the history data processing device 500 are each connected to a network 5 and configured to be able to communicate via the network 5.

[0025] Terminal device 100 is an information device, such as a smartphone or tablet terminal device, that includes a camera CAM (image capture device) and a computer capable of acquiring images using the camera CAM. Camera CAM is a collective term for, for example, camera CAM1 and camera CAM2. Terminal device 100 may include both camera CAM1 and camera CAM2, or may include at least one of them depending on the application. In the following description of this embodiment, camera CAM1 will be simply referred to as camera CAM. Terminal device 100 collects images of a subject (user US2) by executing predetermined application software (hereinafter simply referred to as a program) in response to operations by user US1. The program for utilizing the camera function in a smartphone or tablet terminal device may be configured as a native type that can function on a computer alone. In the following description, a tablet-type terminal device will be used as an example. 2B, the display unit of the terminal device 100 displays a facial image (image USIMG) of the user US2 captured by the camera CAM. The terminal device 100 uses this facial image to detect facial features of the user US2.

[0026] The administrator terminal 200 is an information device including, for example, a smartphone, a tablet terminal device, or a personal computer, and receives operations from the administrator of the above programs, and registers the target programs in the management device 300 in accordance with the operations.

[0027] The management device 300 is a server device managed by, for example, a telecommunications carrier, and receives the target program posted from the administrator terminal 200 and stores it so that it can be viewed by people with predetermined authority. For example, the management device 300 receives a request from the terminal device 100 and permits download of a program in response to the request. In response to the permission for download, the management device 300 transmits the target program to the terminal device 100.

[0028] The history data processing device 500 communicates with both the terminal device 100 and the management device 300, and is able to return information to the terminal device 100 in response to a request from the terminal device 100. In this embodiment, the history data processing device 500 may be omitted. The history data processing device 500 will be described later.

[0029] A more specific example of using the terminal device 100 will be described below.

[0030] Measurement points according to the embodiment will be described with reference to FIGS. 3A to 3E. 3A to 3E are diagrams for explaining measurement points according to the embodiment. Fig. 3A shows an example of a display at the start of measurement, and Fig. 3B shows the coordinate system established by the initialization process.

[0031] At the start of measurement shown in FIG. 3A, a reference line HL is displayed across the screen on the display unit of the terminal device 100. A face image USIMG of the user US2 captured by the camera CAM of the terminal device 100 is displayed on the display unit thereof. A marker MK is placed on the face of the user US2 within a range that can be photographed by the camera CAM. This marker MK is used as an index for obtaining the size of constituent parts within a face image. A color label (sticker) or the like may be used as the marker MK. The image shown in Fig. 3A is an example in which a round sticker with a diameter of 8 mm is used.

[0032] For example, user US1 checks the image of user US2's face displayed on the display unit of terminal device 100 and adjusts the positional relationship between the face of user US2 and terminal device 100. For example, user US1 moves terminal device 100 close to user US2 so that the width of the image of the face fits on the screen, and determines the position of user US2's face so that the center of the left and right of the image of the face is roughly close to the center of the left and right of the display unit of terminal device 100. Note that the adjustment of the center of the left and right of the image of the face does not need to be determined precisely.

[0033] Referring to FIG. 3B, the determination of a coordinate system based on the reference line HL shown in FIG. 3A is shown. The user US1 adjusts the positional relationship of the terminal device 100 and the angle of the camera CAM around the optical axis so that the images of the inner corners of the left and right eyes (internal angles) of the user US2 overlap the reference line HL, and acquires the image. For example, this image is used for measurement when the eyes are open. The details of this process will be described later.

[0034] In the image captured as described above, the reference line HL passes through the position (e.g., near the inner corner of the eye) of the image of the measurement site (e.g., the left and right eyes) of the subject user US2. This reference line HL is used as the reference axis of the two-dimensional coordinate system. A reference line in the vertical direction that is paired with this reference line HL is determined and used as the reference axis of the two-dimensional coordinate system. For example, a vertical reference line CL perpendicular to the reference line HL is placed at the center of the image of the face of user US2, using the positions near the inner corners of the eyes of the user US2 as reference points. The positions near the inner corners of the eyes of the user US2 are denoted as (x2,0) and (x1,0), respectively, and the intersection of the horizontal reference line HL and the vertical reference line (axis CL) is set as the origin O of the two-dimensional coordinate system. As a result, the distance from the reference line HL to the measurement point is equal to the distance between the reference line HL and the horizontal line that passes through the measurement point. The vertical distance between any two measurement points is equal to the distance between the horizontal lines passing through those measurement points. This calculation can be performed indirectly using the distance between the reference line HL and the horizontal line passing through the measurement points.

[0035] To put it another way. By executing a predetermined program, the terminal device 100 can use the reference line HL and the facial center line perpendicular to the reference line HL as axes (axis CL) of a two-dimensional coordinate system, and detect the axial distance between two points of the measurement object using index values ​​on the facial center line.

[0036] Similarly, the terminal device 100 executes a predetermined program to generate a reference line to be used as an axis of a two-dimensional coordinate system and display the reference line HL superimposed on an image of the measurement site of the subject, user US2. Furthermore, by executing a predetermined program, the terminal device 100 may capture an image of the measurement site of the subject (user US2) with the camera CAM at a predetermined timing within the period in which the reference line HL is displayed, and obtain image data including an image of the measurement site of the subject (user US2) and an image of a marker of a predetermined size.

[0037] Figure 3C shows an example of measurement points when the eyes are open. As shown in Figure 3C, the measurement targets when the eyes are open include the pupil center (A), upper eyelid margin (B: upper eyelid margin), double eyelid line (C: double eyelid line), lower eyebrow margin (D: line below the eyebrow), and lower eyelid margin (L: lower eyelid margin) of the subject (user US2). Figure 3D shows an example of measurement points when the eyes are closed. As shown in Figure 3D, the measurement targets when the eyes are closed include the subject's (user US2) upper eyelid margin (E: upper eyelid margin), double eyelid line (F: double eyelid line), or lower eyebrow margin (G: line below the eyebrow). Figure 3E shows an example of measurement points when the eyes are open and the eyes are turned up. As shown in Figure 3E, the measurement targets when the subject (user US2) has their eyes open and their face facing forward, looking upward, include the pupil center (H), upper eyelid margin (I: upper eyelid margin), or lower eyebrow margin (J: line below the eyebrow). The terminal device 100 may store the position information of the measurement points shown in FIGS. 3C, 3D, and 3E in a table shown in FIG. 5, which will be described later.

[0038] Tables for storing information on positions and distances between them according to the embodiment will be described with reference to FIGS. Fig. 4 is a diagram illustrating a table for storing position information for defining two-dimensional coordinates in an embodiment. As shown in Fig. 4, the position information of two points, the right inner corner of the eye (right inner corner of the eye) and the left inner corner of the eye (left inner corner of the eye), detected from a face image, is indicated by x1 and x2. The point between these two points (center) is indicated by x0. The table shown in Fig. 4 stores the above position information.

[0039] FIG. 5 is a diagram illustrating a table for storing position information of a measurement target according to the embodiment. As shown in Fig. 5, position information for each detection target in a face image is stored. The position information for the detection targets stored in the table shown in Fig. 5 includes information such as the lower eyelid margin (edge ​​of the lower eyelid), pupil center (pupil center line), upper eyelid margin (edge ​​of the upper eyelid), double eyelid line (double eyelid line), and lower eyebrow edge (line below the eyebrow). In addition to the above detection targets, the positions of the marker MK include the lower end and upper end of the marker, and this position information is also stored in this table.

[0040] The above position information is the detected values ​​of the respective positions shown in FIGS. 3B to 3E. The measurement conditions include eyes open and eyes closed, and the gaze direction when the eyes are open includes looking straight ahead and looking upward. The letters "A" through "J" in this table correspond to the horizontal lines in Figures 3C-3E.

[0041] FIG. 6 is a diagram illustrating a table for storing information on evaluation results related to ptosis according to the embodiment. As shown in FIG. 6, the evaluation items stored in this table may include, for example, the following items related to ptosis: For example, palpebral fissure height, double eyelid width when eyelids are open, MRD-1 (margin reflex distance-1: eyelid margin corneal reflex distance), LF (levator function), distance from the center of the pupil to the lower edge of the eyebrow, distance from the eyelid margin to the lower edge of the eyebrow when eyelids are closed, and double eyelid width when eyelids are closed.

[0042] The palpebral fissure height is the distance from the upper eyelid margin to the lower eyelid margin. Using the facial image of the subject looking straight ahead with their eyes open as shown in Figure 3C, the difference (CX) between measurement results C and X can be applied.

[0043] The open eyelid width is literally the width of the double eyelid when the eyelids are open. Using the face image of the face in the frontal view with the eyes open shown in Figure 3C above, the difference (CB) between measurement result C and measurement result B can be applied.

[0044] MRD-1 (margin reflex distance-1) is the distance from the center of the pupil to the upper eyelid margin in a relaxed state. For example, using the face image in the state of frontal gaze with eyes open shown in Figure 3C, the difference (B-A) between measurement result B and measurement result A can be applied.

[0045] LF (levator function) refers to the amount of activity of the upper eyelid when the subject looks down and when the subject looks up (upward). Note that downward gaze may be replaced with closed eyes. For example, using the face image with the eyes open and looking up shown in FIG. 3E and the face image with the eyes closed shown in FIG. 3D, LF is calculated using the difference between measurement results J and H and the difference between measurement results G and E. For example, the calculation result of ((JH) - (GE)) may be applied to the above calculation. Although two facial images taken under different conditions are used, the position of measurement result J in FIG. 3E and the position of measurement result G in FIG. 3D are the same, and therefore, although differences in position due to facial expression and the influence of measurement errors are included, they can be considered to be approximately the same value. This approximation allows the difference (EH) between measurement result E and measurement result H to be applied.

[0046] As the distance from the center of the pupil to the lower edge of the eyebrow, for example, the difference (DA) between measurement result D and measurement result A can be applied using the face image in the state of looking straight ahead with eyes open as shown in Figure 3C above.

[0047] As the distance from the eyelid edge to the lower edge of the eyebrow when the eyelid is closed, for example, the difference (GE) between measurement result G and measurement result E can be applied using the face image with the eyes closed shown in FIG. 3D described above.

[0048] As the double eyelid width when the eyelids are closed, for example, the difference (FE) between the measurement result F and the measurement result E can be applied using the face image in the closed eye state shown in FIG. 3D described above.

[0049] The values ​​of the above items are obtained as a result of calculation processing by the calculation processing unit 120 when the computer of the terminal device 100 executes a program, for example.

[0050] In addition to the above-mentioned numerical information items, the table shown in Figure 6 may also include items that determine the subject's condition and indicate it as binary information. Such binary information may be positive (+) or negative (-). Items indicated by such binary information may include information on the results of diagnoses by a doctor, such as Bell's phenomenon, MG, blepharospasm, mass lesions, diplopia, HCL, cataracts, glaucoma, eye movement abnormalities, anisocoria, and other items. For example, these items store information on results identified according to predetermined criteria. Although the evaluation items suitable for evaluating the severity of ptosis symptoms have been exemplified, the evaluation items are not limited to these and may be changed as appropriate depending on the symptoms to be evaluated.

[0051] (Terminal Device) An example of the terminal device 100 etc. according to the embodiment will be described with reference to FIGS. 7A and 7B.

[0052] FIG. 7A is a configuration diagram of the terminal device 100 and the history data processing device 500 according to the embodiment. FIG. 7B is a configuration diagram of the administrator terminal 200 according to the embodiment.

[0053] The terminal device 100 includes, for example, an acquisition unit 110, a processing unit 120, a control unit 130, an output unit 140, a communication unit 150, and a storage unit 160.

[0054] The acquisition unit 110 acquires an image (face image) of the subject and temporarily adds it to the storage unit 160.

[0055] The acquisition unit 110 is configured to include, for example, a camera CAM (image sensor). Camera CAM1 of the acquisition unit 110 is a camera that can capture images of the area behind the surface of the terminal device 100 on which the display unit is provided, and may be called the "out-camera" of the terminal device 100. Camera CAM2 of the acquisition unit 110 is a camera that can capture images of the area near the surface of the terminal device 100 on which the display unit is provided, and may be called the "in-camera" of the terminal device 100. In this embodiment, camera CAM2 is not used. An example in which camera CAM2 is used will be described later. User US1 uses the camera CAM (imaging unit) of the acquisition unit 110 to capture the face of user US2 who is near the terminal device 100 and acquires the image information. The operation of the acquisition unit 110 may be entrusted to a third party other than users US1 and US2. By using the camera CAM of the acquisition unit 110, it is possible to obtain image information of a quality that allows the facial features of user US2 to be detected without using a camera other than the terminal device 100.

[0056] The control unit 130 controls the photographing by the acquisition unit 110 (imaging unit) and the display by the output unit 140 (display unit) to acquire a facial image of the subject user US2 and display it as a moving image on the output unit 140. The control unit 130 generates a reference line to be used as the axis of the two-dimensional coordinate system and displays the reference line on the output unit 140, superimposed on an image of the measurement site of the subject user US2. In this state, the user US1 adjusts the size, position, tilt, etc. of the facial image in the displayed image of the user US2. The control unit 130 captures an image of the measurement site of the user US2 (subject) at a predetermined timing within the period during which the reference line is displayed on the output unit 140, and acquires image data including an image of the measurement site of the user US2 (subject) and an image of a marker of a predetermined size.

[0057] The arithmetic processing unit 120 performs predetermined processing on the image information of the object acquired by the acquisition unit 110. As the predetermined processing, the arithmetic processing unit 120 of this embodiment determines two-dimensional coordinates on an image including an image of the measurement site of the subject user US2 and an image of a marker MK of a predetermined size, the axis of which is a reference line passing through the position of the image of the measurement site. The arithmetic processing unit 120 may use the two-dimensional coordinates to detect the distance in the direction of the axis CL between two points on the measurement object at the measurement site from the image including the image of the measurement site and the image of the marker MK.

[0058] The output unit 140 includes a display unit that displays various images such as an acquired face image, an image showing an operation procedure, a calculation result, etc. The display unit may be provided with an operation reception unit that can be used as a touch panel.

[0059] The communication unit 150 includes a communication interface for communicating with the history data processing device 500 and the management device 300 via a network.

[0060] The storage unit 160 includes, for example, a semiconductor memory element. The terminal device 100 may be configured as described above.

[0061] The flow from application to delivery in this embodiment will be described with reference to FIG. FIG. 8 is a diagram for explaining the detection process according to the embodiment.

[0062] The program related to the detection process of the embodiment is managed by a specific administrator with respect to the content of the process and whether or not it is distributed. For example, when a program becomes available for distribution, the administrator terminal 200, which executes the registration application process (S121), sends the program to the management device 300 via the network. The management device 300 registers the program through the program registration process (S131). This registration makes the program available for download.

[0063] (Advance preparation) The user US1 downloads a program relating to the detection process from the management device 300 in advance in order to execute the program on the terminal device 100. For example, the terminal device 100 requests a download (download request) from the management device 300 (S112). The management device 300 approves the request (S132) and executes a transmission process for the program (S133). The terminal device 100 receives the program and executes an installation process (S113). As a result, an icon associated with the program is added to the home screen of the terminal device 100, and the program can be executed on the terminal device 100. By tapping on this icon, the program starts to be executed. The management device 300 may also perform the billing process (S134) according to the download.

[0064] (Detection process performed by terminal device 100) Next, the processing of the terminal device 100 according to the embodiment will be described. The user US1 taps the icon displayed on the home screen of the terminal device 100 to start the program on the terminal device 100 (S115), and starts its execution.

[0065] The terminal device 100 executes the program to perform a process of detecting the positions of the components (S116). The terminal device 100 displays an operation guide based on the program to show the measurement procedure, and performs measurements of the predetermined detection items in order. The series of processes ends when the required detection items have been measured or in response to a request for interruption from the user US1.

[0066] The sending of measurement results (S117), registration (S157), generation of an inference model (S158), and updating of the inference model (S118) will be described later.

[0067] Next, the "processing for detecting the position of a component" of the terminal device 100 will be described in detail with reference to Fig. 9. Fig. 9 is a flowchart of the detection process according to the embodiment. As described above, the control unit 130 starts the program in response to the operation of the user US1 (S115), and then causes the arithmetic processing unit 120 to perform the "process of detecting the positions of the components" (S116).

[0068] For example, the control unit 130 activates a "camera image display function" in the "process for detecting the position of the component" (S1161). The "camera image display function" is a function for displaying an image captured by the camera CAM of the acquisition unit 110 of the terminal device 100 on the display unit of the output unit 140, for example.

[0069] The calculation processing unit 120 generates a reference line HL to be used as an axis of the two-dimensional coordinate system, and causes the output unit 140 to display the reference line HL so that it overlaps the image of the subject user US2 (image of the measurement site) (S1162).

[0070] In this state, the user US1 adjusts the distance from the terminal device 100, the inclination of the terminal device 100, etc. in order to adjust the position and inclination of the face of the user US2 (S1163).

[0071] The control unit 130 performs a process for allowing the user US1 to select a measurement mode (S1164). For example, the selectable measurement modes include a classification of eyes open and eyes closed that defines the measurement conditions, and a classification of gaze directions when the eyes are open, that is, frontal gaze and upward gaze. The control unit 130 performs measurement in the measurement mode designated by the user US1 in the following procedure.

[0072] The control unit 130 takes a photograph using the camera CAM (S1165), and displays the still image obtained by taking the photograph on the display unit of the output unit 140 (S1166). The control unit 130 adds the image information of the captured still image to the storage unit 160 (S1167). The arithmetic processing unit 120 sets a measurement target area in the face image displayed on the display unit of the output unit 140 (S1168), and after setting the measurement target area, measures the position (S1169).

[0073] The calculation processing unit 120 may perform the setting of the measurement target location (S1168) and the measurement of the position (S1169) in accordance with the procedure of the measurement mode designated by the user US1.

[0074] (Determining the feature amount to be detected) The determination of the feature amount of the detection target in the embodiment will be described in more detail with reference to Figures 10 and 11. The application exemplified here is intended to be applied to a case where the condition of a person suffering from ptosis is detected.

[0075] In the first technique, a person (for example, user US1) reads an image and sets feature points to be detected. The second method uses image recognition processing to read an image and set feature points of the detection target. In this embodiment, the first method will be described, and the second method will be described later.

[0076] 10A and 10B are explanatory diagrams relating to a process for determining a feature amount of a detection target according to the first method of the embodiment, and show examples of images displayed on the display unit of the terminal device 100. Icons for operating the terminal device 100 are displayed at the top of the display image shown in each drawing, and a facial image to be analyzed is displayed below them. In the case of this first method, the process for detecting the feature amount is semi-automatic based on the operation of the user US1 as described below.

[0077] The operation icons are associated with functions for switching between displays and processes. For example, operation icons include "Full" for displaying the entire facial image of user US2, "Left / Right" for selecting the left or right eye, and "OK" for proceeding to the next step.

[0078] When the control unit 130 of the terminal device 100 (hereinafter, sometimes simply referred to as the terminal device 100) detects an operation on the "whole" icon or is in the initial state, it displays the entire face image of the user US2. In this case, the display unit displays the face image (both eyes) of the user US2 with their eyes open and each axis of the two-dimensional coordinate system.

[0079] The terminal device 100 accepts an operation by the user US1 for aligning the image and enlarging the display, and enlarges and displays a portion of the facial image of the user US2 to be analyzed. For example, when the terminal device 100 detects an operation of the "left / right" icon, it selects the left eye or the right eye of the user US2. By repeating this operation, the image alternates between the image in FIG. 10(a) and the image in FIG. 10(b).

[0080] Next, the terminal device 100 detects the operation of the "OK" icon and enlarges and displays the selected eye. The example shown in FIG. 11 is an enlarged view of one eye of the user US2 in an open eye state. FIG. 11 is a diagram for explaining processing for one eye in an open eye state according to the embodiment. 11(a) to 11(c) show examples of images displayed on the display unit of the terminal device 100. In FIG. At this time, the display unit displays one eye of the user US2 in an open state, each axis of the two-dimensional coordinate system, and an auxiliary line used to determine the feature amount. The position of the one eye in an open state on the two-dimensional coordinate system is determined, and the relative position cannot be adjusted. In contrast, the auxiliary line used to determine the feature amount can be moved on the two-dimensional coordinate system in a direction perpendicular to the extension direction of the auxiliary line (for example, in the up and down direction of the image of the face).

[0081] For example, while looking at the face image of user US2 displayed on the display unit of terminal device 100, user US1 places his / her finger on the position of an auxiliary line (A, B, C, or D) also displayed on the display unit. User US1 slides the position of his / her finger touching the display unit (touch panel) in the vertical direction of the face image, thereby moving the auxiliary line in the vertical direction of the face image.

[0082] For example, as shown in FIG. 11(a), user US1 moves auxiliary line B in the vertical direction of the image of user US2's face by a slide operation, adjusts it so that it touches the lower limit of the eyelid when the eyes are open, and releases his / her finger at an appropriate position. Also, as shown in FIG. 11(b), user US1 moves auxiliary line C in the vertical direction of the image of user US2's face by a slide operation, adjusts it so that it touches the double eyelid contour when the eyes are open, and releases his / her finger at an appropriate position. Also, as shown in FIG. 11(c), user US1 moves auxiliary line D in the vertical direction of the image of the face by a slide operation, adjusts it so that it touches the lower end near the center of the eyebrow contour, and releases his / her finger at an appropriate position. Also, user US1 moves auxiliary line A in the vertical direction of the image of the face by a slide operation, adjusts it so that it passes through the center of the pupil, and releases his / her finger at an appropriate position. After placing each auxiliary line as described above, the user US1 operates the "OK" icon.

[0083] The calculation processing unit 120 of the terminal device 100 identifies the position of each auxiliary line placed by the operation of the user US1. For example, the terminal device 100 identifies the position of each auxiliary line as the intercept value of a coordinate axis of a two-dimensional coordinate system. As the image is enlarged, the scale of the coordinate axes is also enlarged. Therefore, it is advisable to adjust the coefficient so that the intercept value corresponding to the position of each auxiliary line becomes an appropriate value.

[0084] The terminal device 100 stores the position of each of the read auxiliary lines in the table shown in FIG. Following the above, the terminal device 100 may also perform similar processing on images with eyes closed and images with eyes looking up. In this way, the feature amount that can be read from the facial image of the user US2 can be quantified by the operation of the user US1.

[0085] According to the embodiment, the program causes a computer including an acquisition unit 110 (imaging unit) and an output unit 140 that displays on a display unit an image of the measurement site of the subject captured by the acquisition unit 110. The program includes the steps of: determining two-dimensional coordinates on an image including the image of the measurement site of the subject and an image of a marker MK of a predetermined size, with a reference line HL as its axis passing through the position of the image of the measurement site; and detecting, using the two-dimensional coordinates, the distance between two points on the subject at the measurement site in the direction of the axis CL from the image including the image of the measurement site and the image of the marker MK. This makes it possible to improve the reproducibility when quantifying the positional relationship of the components of the measurement target.

[0086] The measurement sites of the subject include the left and right eyes of the subject's face. The reference line HL in the displayed image is located near the inner corner of the eye. Using the above program, the reference line HL can be used as one axis of a two-dimensional coordinate system to detect the distance along the axis CL between two points on the measurement object from the image including the image of the subject's face.

[0087] This program may include a step of generating scale information for an image including an image of the subject's face from the size of the image of the marker MK reflected in the image, and using the generated scale information to detect the distance in the direction of the axis CL between two points on the measurement target.

[0088] Furthermore, this program may include a step of deriving the distance from the reference line HL to the image of the measurement object as the distance between the reference line HL and a line segment parallel to the reference line HL.

[0089] In addition, this program includes steps of placing the reference line HL on the image of the subject's eye, extracting the position of the subject's inner corner of the eye, and displaying an indicator at the position of the subject's inner corner of the eye, thereby enabling the condition of the subject's eyes to be quantified with good reproducibility.

[0090] The program is configured to identify from the image the subject's pupil center (pupil center line), upper eyelid margin (edge ​​of the upper eyelid), double eyelid line (double eyelid line), or lower eyebrow margin (line below the eyebrow).

[0091] As described above, the terminal device 100 (display device) includes the acquisition unit 110 including at least a camera CAM (imaging unit), the arithmetic processing unit 120, the control unit 130, and the output unit 140. The output unit 140 causes the display unit to display the image of the measurement site of the subject captured by the camera CAM. The control unit 130 controls the capturing of images by the camera CAM and the display by the display unit in the output unit 140. The calculation processing unit 120 performs predetermined calculations. For example, the control unit 130 generates a reference line to be used as the axis of the two-dimensional coordinate system, displays the reference line superimposed on an image of the subject's measurement site, and captures an image of the subject's measurement site at a predetermined timing during the period in which the reference line is displayed to obtain image data including an image of the subject's measurement site and an image of a marker of a predetermined size. The calculation processing unit 120 determines two-dimensional coordinates with a reference line passing through the position of the image of the measurement site on the image containing an image of the measurement site of the subject and an image of a marker of a predetermined size as its axis, and uses the two-dimensional coordinates to detect the axial distance between two points of the measurement subject at the measurement site from the image containing the image of the measurement site and the image of the marker. Such a terminal device 100 (display device) can improve the reproducibility when quantifying the positional relationship of the components of the measurement target.

[0092] In the above embodiment, the measurement items and measurement method are exemplified as examples in which two measurement points on the measurement site are predetermined, but the present invention is not limited to this, and the program of the terminal device 100 (display device) may be configured to measure the distance between two points on the image of the face of user US2 that are arbitrarily designated by user US1. In this case, as in the above embodiment, the distance of the actual measurement site on the face of user US2 may be calculated from the relationship between the size of the image of the marker on the image of the face of user US2 and the distance between the arbitrarily designated two points.

[0093] The above-described terminal device 100 (display device) can be used in ophthalmology treatment, for example, treatment for ptosis. When treating ptosis, there are 7-8 items to record for each eyelid. In the comparative example, the doctor measures each item in order using a caliper or similar tool and records the measurements in the electronic medical record. This requires an accuracy of about 0.5 mm. This process is repeated throughout the consultation. In contrast, by using the above-mentioned terminal device 100 (display device), several images are taken using the camera CAM of the terminal device 100 (display device), and auxiliary lines for measurement are sequentially placed at the measurement target locations of the eye images on the images.The terminal device 100 (display device) uses the results of this placement to calculate the distances of the measurement locations for the items to be recorded in the symptom record, thereby obtaining quantified data.

[0094] (Second embodiment) A process for determining a feature amount of a detection target according to the second method of the embodiment will be described with reference to FIGS. Fig. 12 is an explanatory diagram of a process for determining a feature amount of a detection target according to a second method of the embodiment, and Fig. 13 is a diagram for explaining a classification process according to the embodiment.

[0095] 12 and 13(a) to (c) show examples of images displayed on the display unit of the terminal device 100. FIG. Icons for operating the terminal device 100 are displayed at the top of the display image shown in each drawing, and a facial image to be analyzed is displayed below them. In the case of the second method, the following is an example of automating the identification process for detecting feature quantities. For example, the terminal device 100 displays, on its display unit, a face image to be analyzed and icons for operation.

[0096] The terminal device 100 accepts an operation by the user US1 to select an image and start classification processing, and displays the face image to be analyzed. The example shown in Fig. 12 displays an example of a face image with eyes open. This display shows the eyes (both eyes) in their open state and each axis of the 2D coordinates. The position of the eyes in their open state on the 2D coordinates is fixed, and the relative position cannot be adjusted.

[0097] As shown in FIGS. 12 and 13, the terminal device 100 automatically extracts the positions of the eyes based on two-dimensional coordinates. 13(b), the terminal device 100 identifies the boundary between the eyelid and the eye, double eyelids, the outline of the eyebrows, the iris, the pupil, etc., by two-dimensional differentiation processing (contour extraction processing) of the face image, area classification processing, etc. There are no limitations on the algorithms applied to the calculation processing such as the contour extraction processing and area classification processing, and known algorithms may be applied as appropriate.

[0098] As shown in FIG. 13(c), the terminal device 100 of this embodiment quantifies the positional relationship of the components from the above detection results, using the detection results of the boundary between the eyelid and the eye, the double eyelid, the outline (lower limit) of the eyebrows, and the center position of the pupil. For example, the terminal device 100 may estimate the center position of the pupil from the detection results of the iris and pupil. The terminal device 100 obtains a line segment (perpendicular line) extending upward from the estimated pupil position to the face. The intersections of this line segment (perpendicular line) with the detection results of the boundary between the eyelid and the eye, the double eyelid, and the outline (lower limit) of the eyebrows may be set as feature points. A line segment extending horizontally from this feature point toward the center of the face is drawn. The line segment extending horizontally from this feature point toward the center of the face corresponds to the "auxiliary line" in the first step above. Therefore, according to the second means, the adjustment of the position of the auxiliary line can be automated using the image processing described above.

[0099] (Modification of the second embodiment) In the second embodiment, a case has been described in which an example of a predetermined rule-based identification method is applied to the identification process of the second means. Alternatively, AI (artificial intelligence) may be used to extract features. For example, the segmentation process may be implemented using a trained model that has been trained to identify objects in the computational processes such as the contour extraction process and region classification process. This makes it possible to generate features directly or indirectly from a face image by identifying the object from the results of region segmentation by the segmentation process, instead of rigorously extracting detailed contours, the centers of pupils, and the like using the rule-based classification method described above.

[0100] When generating the trained model, existing data stored in the history data processing device 500 may be used as training data. A combination of image data of a face image of a subject before the classification process and data after the classification process shown in the first or second embodiment may be applied to this training data.

[0101] For example, as shown in FIG. 7A above, the history data processing device 500 includes an information acquisition unit 510, a calculation processing unit 520, a communication unit 530, and a storage unit 540.

[0102] The information acquisition unit 510 acquires information generated by the management device 300, the terminal device 100, etc., and temporarily adds the information to the storage unit 540.

[0103] The arithmetic processing unit 520 performs a learning process for a model for extracting facial features based on the information generated by the terminal device 100, and adds the model to the storage unit 540. The arithmetic processing unit 520 may update the learned model as needed.

[0104] The communication unit 530 includes a communication interface for communicating with the terminal device 100 and the management device 300 via a network. The storage unit 160 includes a semiconductor memory, a magnetic recording device, and the like. The history data processing device 500 configured as described above collects and manages information collected from the terminal device 100.

[0105] 8, when the terminal device 100 transmits the measurement result (S117), the history data processing device 500 acquires it and registers it in the storage unit 160 (S157). This is carried out as appropriate.

[0106] Furthermore, the calculation processing unit 520 generates an inference model using the collected measurement results (S158). This process may apply a technique such as machine learning. The calculation processing unit 520 distributes the generated inference model to the terminal device 100. In response to this, the terminal device 100 updates the distributed inference model (trained model) (S118). This allows the terminal device 100 to use the inference model to extract features of each part of the face.

[0107] It should be noted that, instead of the terminal device 100 performing feature extraction, the calculation processing unit 520 may use an inference model to perform feature extraction and notify the terminal device 100 of the results.

[0108] (Third embodiment) An example of a different usage method from the above embodiment will be described with reference to FIGS. Fig. 14 is a schematic configuration diagram of an information processing system 1A according to the third embodiment, and Fig. 15 is a diagram for explaining the information processing system 1A according to the third embodiment. The user US3 in this embodiment serves as both the user US1 and the user US2 in the above-described embodiment, and the users US1 and US2 in the above-described embodiment may be read as the user US3.

[0109] The terminal device 100 executes a program in response to an operation by the user US3, thereby collecting images of the subject (the user US3 himself / herself). In this case, the terminal device 100 uses the camera CAM2 as the camera CAM. In this embodiment, as described above, the user U3 to be analyzed can personally operate the terminal device 100. Note that adjustment of the angle around the optical axis of the camera CAM2 may be performed instead of or in combination with adjustment of the angle of the user's own head.

[0110] (Effects common to all embodiments) According to the terminal device 100 (program) having the above configuration, a computer including an imaging unit and an output unit that displays an image of the subject's measurement site captured by the imaging unit on a display unit executes the steps of: determining two-dimensional coordinates on an image including the subject's measurement site and an image of a marker of a predetermined size, with a reference line passing through the position of the image of the measurement site as its axis; and using the two-dimensional coordinates to detect the axial distance between two points on the measurement object at the measurement site from the image including the image of the measurement site and the image of the marker. This allows determining two-dimensional coordinates on an image including the subject's measurement site and an image of a marker of a predetermined size, with a reference line passing through the position of the image of the measurement site as its axis. It is preferable to execute the step of using the two-dimensional coordinates to detect the axial distance between two points on the measurement object at the measurement site from the image including the image of the measurement site and the image of the marker. This improves the reproducibility of the numerical representation of the positional relationship of the components of the measurement object. The terminal device 100 (program) can easily quantify the condition of ptosis in ophthalmology, for example. Furthermore, it can provide an index that enables an objective decision on whether or not surgical treatment should be selected when the symptoms of ptosis have progressed, along with facial images that serve as the basis for that decision. Progress monitoring during ptosis treatment can also be performed by comparing the index value of historical data and the facial images that serve as the basis for that index value with the current index value.

[0111] In a comparative example, a digital camera is used as an imaging system, and a processing device is configured separately from the digital camera. In such a configuration, it becomes difficult to photograph a face by adjusting its position, size, tilt, etc. so that it fits within the image. In contrast to this, by using the terminal device 100 (program) of this embodiment, it is possible to implement an imaging system and a processing system in an integrated device.

[0112] (Other embodiments) The above describes in detail an embodiment of the present invention with reference to the drawings, but the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope that does not deviate from the gist of the present invention. In the above embodiment, a tablet-type device is exemplified as the terminal device 100, but instead of this, it may be configured as a dedicated device with a limited use.

[0113] The information processing system can also be realized by an information processing method in the information processing system and a computer program that executes the corresponding processing.

[0114] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Addendum) (1) In one aspect, the program An imaging unit; a display unit that displays an image of the measurement site of the subject captured by the imaging unit; A computer comprising: A step of determining two-dimensional coordinates with a reference line passing through the position of the image of the measurement site of the subject as an axis in an image including an image of the measurement site and an image of a marker of a predetermined size, and detecting an axial distance between two points of the measurement object at the measurement site from the image including the image of the measurement site and the image of the marker using the two-dimensional coordinates. This is a program for executing the above. (2) In the program described in (1) above, The measurement site of the subject includes the left and right eyes of the subject's face, The reference line in the image is located near the inner corner of the eye, Using the reference line as an axis of a two-dimensional coordinate system, the distance in the axial direction between two points on the measurement object is detected from an image including an image of the face of the subject. It is good to include. (3) The program described in (1) or (2) above, generating scale information of an image including an image of the subject's face from the size of the image of the marker reflected in the image; Detecting an axial distance between two points on the measurement object using the generated scale information. It is good to include. (4) The program according to any one of (1) to (3) above, A step of deriving a distance from the reference line to the image of the measurement object as a distance between the reference line and a line segment parallel to the reference line. It is good to include. (5) The program according to any one of (1) to (4) above, the reference line spans an image of the subject's eye; Extracting the position of the subject's inner corner of the eye; and displaying an indicator at the position of the subject's inner corner of the eye. It is desirable that the above be performed. (6) The program according to any one of (1) to (5) above, The measurement targets when the eyes are open may include any of the subject's pupil center, upper eyelid margin, double eyelid line, or lower eyebrow margin. (7) The program according to any one of (1) to (6) above, The measurement targets when the eyes are closed may include any of the subject's upper eyelid margin, double eyelid line, or lower eyebrow margin (the line below the eyebrow). (8) The program according to any one of (1) to (7) above, The measurement targets when the subject is facing forward, looking upward, and has their eyes open should preferably include any of the center of the pupil, the upper eyelid margin, or the lower eyebrow margin (the line below the eyebrows). (9) The program according to any one of (1) to (8) above, It is preferable that the device executes a step of identifying from the image any of the subject's pupil center, upper eyelid margin, double eyelid line, or lower eyebrow margin (the line below the eyebrow). (10) The program according to any one of (1) to (9) above, The reference line and the facial center line perpendicular to the reference line are used as the axes of the two-dimensional coordinate system, The method may further include detecting an axial distance between two points on the measurement object using an index value on the facial centerline. (11) The program according to any one of (1) to (10) above, generating a reference line to be used as an axis of a two-dimensional coordinate system, displaying the reference line superimposed on an image of the measurement site of the subject, and capturing an image of the measurement site of the subject at a predetermined timing within a period during which the reference line is displayed to obtain image data including an image of the measurement site of the subject and an image of a marker of a predetermined size; It is desirable that the above be performed. (12) The program according to any one of (1) to (11) above, taking an image of a measurement site including the subject's eyes and recording data indicating the degree of the subject's ptosis symptom; The program is for executing a step of making it possible to view or compare data on the degree of symptoms of ptosis of the subject with historical data. (13) A display device according to one embodiment includes: An imaging unit; a display unit that displays an image of the measurement site of the subject captured by the imaging unit; and a control unit that controls the imaging unit to capture an image and the display unit to display an image; a calculation processing unit that performs a predetermined calculation; Equipped with The control unit generating a reference line to be used as an axis of a two-dimensional coordinate system, displaying the reference line superimposed on an image of the measurement site of the subject, and capturing an image of the measurement site of the subject at a predetermined timing within a period during which the reference line is displayed to obtain image data including an image of the measurement site of the subject and an image of a marker of a predetermined size; The arithmetic processing unit The reference line passing through the position of the image of the measurement site of the subject in an image including an image of the measurement site and an image of the marker of the specified size is used as the axis of the two-dimensional coordinate system, and the axial distance between two points of the measurement object at the measurement site is detected from the image including the image of the measurement site and the image of the marker. Display device. (14) An information processing system according to one aspect includes: An imaging unit; a display unit that displays an image of the measurement site of the subject captured by the imaging unit; An information processing system using a computer comprising: In the computer, In an image including an image of the subject's measurement site and an image of a marker of a predetermined size, two-dimensional coordinates are determined with a reference line passing through the position of the image of the measurement site as an axis, and the two-dimensional coordinates are used to detect the axial distance between two points of the measurement object at the measurement site from the image including the image of the measurement site and the image of the marker. It is recommended to run the following. [Explanation of symbols]

[0115] 1, 1A...Information processing system 100...Terminal device 110... Acquisition unit (including imaging unit (camera CAM)) 120...Calculation processing unit 130...Control unit 140...Output section 160...Storage section 200: Administrator terminal 300…Management device US1, US2, US3...Users

Claims

1. An imaging unit; a display unit that displays an image of the measurement site of the subject captured by the imaging unit; A computer comprising: determining two-dimensional coordinates with a reference line passing through the position of the image of the measurement site of the subject as an axis in an image including an image of the measurement site and an image of a marker of a predetermined size, and detecting an axial distance between two points of the measurement object at the measurement site from the image including the image of the measurement site and the image of the marker using the two-dimensional coordinates; A program to execute.

2. The measurement site of the subject includes the left and right eyes of the subject's face, The reference line in the image is located near the inner corner of the eye, Using the reference line as an axis of a two-dimensional coordinate system, the distance in the axial direction between two points on the measurement object is detected from an image including an image of the face of the subject. The program according to claim 1 , comprising:

3. generating scale information of an image including an image of the subject's face from the size of the image of the marker reflected in the image; Detecting an axial distance between two points on the measurement object using the generated scale information. The program according to claim 2 , comprising:

4. A step of deriving a distance from the reference line to the image of the measurement object as a distance between the reference line and a line segment parallel to the reference line. The program according to claim 1 , comprising:

5. the reference line spans an image of the subject's eye; extracting a position of the subject's internal angle; and displaying an indicator at the position of the subject's internal canthus. The program according to claim 1 , comprising:

6. The measurement target when the eyes are open includes any of the subject's pupil center, upper eyelid margin, double eyelid line, and lower eyebrow edge. The program according to claim 1.

7. The measurement target when the eyes are closed includes the upper eyelid margin, double eyelid line, or lower eyebrow edge of the subject. The program according to claim 1.

8. The measurement target when the subject is facing forward and looking upward with their eyes open includes the pupil center, upper eyelid edge, or lower eyebrow edge of the subject. The program according to claim 1.

9. Identifying from the image any one of the pupil center, upper eyelid margin, double eyelid line, and lower eyebrow margin of the subject. The program according to claim 1 for executing the above.

10. The reference line and a facial center line perpendicular to the reference line are used as axes of the two-dimensional coordinate system, and detecting an axial distance between two points on the measurement object using an index value on the facial centerline. The program according to any one of claims 3 to 9.

11. generating a reference line to be used as an axis of a two-dimensional coordinate system, displaying the reference line superimposed on an image of the measurement site of the subject, and capturing an image of the measurement site of the subject at a predetermined timing within a period during which the reference line is displayed to obtain image data including an image of the measurement site of the subject and an image of a marker of a predetermined size; The program according to claim 1 for executing the above.

12. taking an image of a measurement site including the subject's eyes and recording data indicating the degree of the subject's ptosis symptom; a step of making it possible to view or compare data on the degree of symptoms of ptosis of the subject with historical data; The program according to claim 1 for executing the above.

13. An imaging unit; a display unit that displays an image of the measurement site of the subject captured by the imaging unit; and a control unit that controls the imaging unit to capture an image and the display unit to display an image; a calculation processing unit that performs a predetermined calculation; Equipped with The control unit generating a reference line to be used as an axis of a two-dimensional coordinate system, displaying the reference line superimposed on an image of the measurement site of the subject, and capturing an image of the measurement site of the subject at a predetermined timing within a period during which the reference line is displayed to obtain image data including an image of the measurement site of the subject and an image of a marker of a predetermined size; The arithmetic processing unit The reference line passing through the position of the image of the measurement site of the subject in the image including the image of the measurement site and the image of the marker of the predetermined size is used as the axis of the two-dimensional coordinate system to detect the axial distance between two points of the measurement object at the measurement site from the image including the image of the measurement site and the image of the marker. Display device.

14. An imaging unit; a display unit that displays an image of the measurement site of the subject captured by the imaging unit; An information processing system using a computer comprising: In the computer, A two-dimensional coordinate system is defined on an image including an image of the subject's measurement site and an image of a marker of a predetermined size, with a reference line passing through the position of the image of the measurement site as its axis, and the two-dimensional coordinate system is used to detect the axial distance between two points on the measurement object at the measurement site from the image including the image of the measurement site and the image of the marker. An information processing system that executes the above.

Images