Information processing device, control method and information processing program

By adjusting the amount of light emitted and image processing technology, the information processing device solves the problem of obtaining high-quality fundus images without professional equipment, enabling efficient fundus observation in remote areas and reducing discomfort for test subjects and image quality issues.

CN123076341APending Publication Date: 2026-11-10NATIONAL CANCER CENTER(JP)
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202580023592.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-02-28
Filing Date
2025-02-27
Publication Date
2026-11-10

Smart Images

  • Figure CN123076341A_ABST
    Figure CN123076341A_ABST
Patent Text Reader

Abstract

The information processing device J includes a dimming unit 32, a still image data acquisition unit P2, and an extraction unit 33. The dimming unit 32 adjusts the amount of light emitted by the light-emitting unit 61 directed towards the subject's eyes. The still image data acquisition unit P2 acquires multiple still image data based on a captured image taken from the fundus of the subject receiving light emitted from the light-emitting unit 61. The extraction unit 33 extracts extracted image data showing the fundus from each of the multiple still image data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to information processing apparatus, control method, and information processing program. Background Technology

[0002] Fundus diseases such as glaucoma, diabetic retinopathy, and macular degeneration often lead to blindness because symptoms appear late, even when treatment begins at the onset of symptoms. However, only ophthalmologists can observe and diagnose the fundus. Therefore, not only in developing countries, but also in remote areas such as mountainous regions and islands, and among the elderly who have limited access to ophthalmologists, there is a significant potential risk of blindness. Unlike anterior eye diseases, which can be treated even in their late stages, the delay in receiving ophthalmologist diagnosis is a problem unique to fundus diseases. Consequently, the demand for telemedicine, enabling fundus observation in areas without ophthalmologists, is increasing.

[0003] To enable remote diagnosis, it is necessary to acquire diagnostically usable images of the fundus of the subject in environments without specialized ophthalmological equipment, and to provide these high-quality images to remote ophthalmologists. To assist in the imaging technique, various adapters have been developed to mount fundus observation lenses onto smartphones with cameras (e.g., International Patent Publication No. 2016 / 154558 and U.S. Patent Application Publication No. 2019 / 0191988). These adapters maintain a consistent positional relationship between the smartphone's camera, the observation lens, and the subject's eye. Existing technical documents Patent documents

[0004] Patent Document 1: International Publication No. 2016 / 154558 Patent Document 2: U.S. Patent Application Publication No. 2019 / 0191988 Summary of the Invention The technical problem that the invention aims to solve

[0005] However, even with traditional adapters, it is difficult to obtain images of quality suitable for fundus observation due to the difference from specialized ophthalmic equipment.

[0006] The embodiments of the present invention were made in view of the following circumstances, and the object is to provide an information processing apparatus, control method and information processing program that can easily acquire fundus images suitable for fundus observation. Technical means to solve the problem

[0007] The information processing device includes a dimming unit, a still image data acquisition unit, and an extraction unit. The dimming unit adjusts the amount of light emitted by a light-emitting unit directed towards the eyes of the subject. The still image data acquisition unit acquires multiple still image data sets based on a captured image of the fundus of the subject receiving light emitted from the light-emitting unit. The extraction unit extracts extracted image data showing the fundus from each of the multiple still image data sets.

[0008] According to embodiments of the present invention, an information processing apparatus, control method, and information processing program are provided that can easily acquire fundus images suitable for fundus observation. Attached Figure Description

[0009] Figure 1 This is a cross-sectional schematic diagram showing the structure of an eye captured by the information processing device according to the first embodiment. Figure 2 This is a schematic diagram showing an example of an adapter used for fundus photography. Figure 3A This is a schematic diagram illustrating an example of a fundus image. Figure 3B This is an explanation Figure 3A The diagram shows cross-sectional views of the eyeball at various locations on the fundus. Figure 4 This is a diagram showing an outline of the system structure of the information processing apparatus according to the first embodiment. Figure 5 This is a block diagram illustrating the hardware structure of the information processing apparatus according to the first embodiment. Figure 6 This is a functional block diagram illustrating an example of the functional structure of the information processing apparatus according to the first embodiment. Figure 7 This is a block diagram illustrating an example of the structure of a camera. Figure 8A It is a graph showing the lengths of various distances when photographing the fundus of a test subject. Figure 8B This is an explanatory diagram illustrating how to photograph a subject several meters away by using the light-emitting part of the information processing device according to the first embodiment. Figure 9 This is a flowchart illustrating an example of fundus imaging processing of the information processing apparatus according to the first embodiment. Figure 10 This is a sequence diagram illustrating an example of fundus imaging processing of the information processing apparatus according to the first embodiment. Figure 11 This is a flowchart illustrating an example of the light intensity adjustment processing of the information processing apparatus according to the fourth embodiment. Figure 12 This is an explanatory diagram illustrating the optical system for capturing the fundus in the information processing apparatus according to the fourth embodiment. Figure 13 This is a diagram showing the design values ​​of the optical system for capturing the fundus in the information processing apparatus according to the fourth embodiment. Detailed Implementation

[0010] (Structure of the information processing device J according to the first embodiment) First, let's explain the structure of the eye as captured by the information processing device J.

[0011] Figure 1 This is a cross-sectional schematic diagram showing the structure of the eye as captured by the information processing device J. The eyeball 70 has a sclera 71, choroid 72, retina 73, vitreous body 74, lens 75, cornea 76, and iris 78. The lens 75 is supported by ciliary bands 77. The expansion or contraction of the iris 78 causes changes in the size of the pupil 79. The retina 73 is called the fundus.

[0012] The macula 82 corresponds to the center of the retina 73. Cells responsible for vision and color vision are concentrated in the macula 82. The center of the macula 82 is the fovea 81. Multiple nerve fibers are distributed in the retina 73, and these nerve fibers converge at a location called the optic disc 84 to form the optic nerve 83.

[0013] Figure 2 This is a schematic diagram showing the appearance of an example of an adapter 90 used for fundus photography. Figure 2 This indicates that an adapter 90, serving as an auxiliary tool for fundus photography, has been installed on a smartphone that functions as a portable information processing terminal 150 for fundus photography. The adapter 90 has a main body 91, an objective lens 92, and a lens support 93 for supporting the objective lens 92. An opening 96 is formed on the main body 91 at the position of the camera 44 of the portable information processing terminal 150. A claw 95 is provided at the lower part of the adapter 90.

[0014] The lens support 93 is supported on the upper part of the main body 91 in a manner that allows it to rotate about the rotation axis 94, and can adjust the optical axis of the imaging lens 52 and the optical axis of the objective lens 92 in the camera 44 of the portable information processing terminal 150. That is, the adapter 90 holds the portable information processing terminal 150 so that the objective lens 92 and the camera 44 are positioned relative to the subject's eye at a predetermined shooting position.

[0015] For use Figure 2The steps for using the adapter 90 and portable information processing terminal 150 to capture static images of the fundus of a subject are explained below. The doctor attaches the main body 91 of the adapter 90 from the top of the portable information processing terminal 150 and hooks the claw 95 onto the lower end of the portable information processing terminal 150. Thus, the adapter 90 is fixed to the portable information processing terminal 150. In a dark room, the doctor operates the portable information processing terminal 150 to put it into shooting mode. The purpose of shooting in a dark room is to maximize the dilation of the subject's pupil 79 to facilitate fundus imaging. At this time, mydriatic eye drops can be instilled into the subject's eyes to dilate the pupil 79 before shooting. With the subject's eye close to or directly facing the objective lens 92, the doctor inputs the shooting instruction into the portable information processing terminal 150, which then emits a flash and captures a static image of the fundus.

[0016] Figure 3A This is a schematic diagram illustrating an example of a fundus image. Figure 3B This is an explanation Figure 3A The diagram shows cross-sectional views of the eyeball 70 at various locations on the fundus. (See attached diagram.) Figure 3A As shown, as a standard, images suitable for fundus observation can include clearly imaged images of the optic disc 84. As another standard, images of clearly imaged images of the retinal vessels 85, composed of arteries 85a and veins 85b, can also be listed. As yet another standard, images of clearly imaged macula 82, to the point where the poles of macula 82 are clearly identifiable, can also be listed. That is, images suitable for fundus observation are images in which at least one of the optic disc 84, retinal vessels 85, and macula 82 is clearly imaged.

[0017] However, using Figure 2 When the adapter 90 and portable information processing terminal 150 photograph the fundus of the subject, sometimes images of usable quality for fundus observation cannot be obtained. The reasons for the difficulty in obtaining images of usable quality for fundus observation are believed to be the following three points.

[0018] (1) When the flash during shooting causes one or both of the subject's eyeballs 70 and eyelids to move frequently, the positional relationship between the camera 44, objective lens 92 and the subject's fundus will change.

[0019] (2) The doctor needs to cooperate with the changes in (1) above, while changing the position of the camera 44 and the objective lens 92, and simultaneously operate the portable information processing terminal 150 to take pictures in parallel.

[0020] (3) If the portable information processing terminal 150 is slightly shaken due to hand tremors or other reasons through the operation of (2) above, the positional relationship between the objective lens 92 and the fundus of the subject will change again.

[0021] When the information processing device J takes a picture of the subject's fundus, it sets the light emission level of the light-emitting unit 61 to be lower than the default value of the portable information processing terminal 1.

[0022] Figure 4 This is a diagram showing the general structure of the information processing device J. The information processing device J has a portable information processing terminal 1, a server 2, a diagnostic terminal 3, and a subject terminal 4 that are communicatively connected to each other via a network 100 such as the Internet.

[0023] Portable information processing terminal 1, diagnostic terminal 3, and subject terminal 4 are smartphones or tablets, etc. Portable information processing terminal 1 is used by medical professionals such as doctors or nurses in facilities such as hospitals or clinics in areas without ophthalmologists. Hereinafter, the user operating portable information processing terminal 1 is a medical professional, but not limited to this. Diagnostic terminal 3 is used by doctors. Subject terminal 4 is used by subjects undergoing eye examinations. Server 2 stores information received from portable information processing terminal 1 and diagnostic terminal 3, and relays inter-terminal communication between portable information processing terminal 1, diagnostic terminal 3, and subject terminal 4.

[0024] Figure 5 This is a block diagram showing the hardware structure of the information processing device J. The portable information processing terminal 1 has a CPU 11, ROM 12, RAM 13, bus 14, input / output interface 15, output unit 16, input unit 17, storage unit 18, communication unit 19, and driver 20.

[0025] CPU 11 performs various processes according to the program recorded in ROM 12 or the program loaded into RAM 13 from storage unit 18. RAM 13 also appropriately stores data required by CPU 11 to perform various processes. CPU 11, ROM 12 and RAM 13 are interconnected via bus 14. Input / output interface 15 is also connected to bus 14.

[0026] The input / output interface 15 is connected to an output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a driver 20. The output unit 16, consisting of a display 41 and a speaker 42, is used to output various information as images and sounds. The input unit 17, consisting of a keyboard and a mouse, is used to input various information. The storage unit 18, consisting of storage elements such as an HDD and an SSD, is used to store various data. The communication unit 19 communicates with other devices via a network 100.

[0027] The drive 20 can be fitted with a removable medium 21, such as a magnetic disk, optical disk, optical disc, or semiconductor memory. Programs read from the removable medium 21 by the drive 20 are stored in the storage unit 18 as needed. Furthermore, the removable medium 21 can also store various data stored in the storage unit 18 in the same manner as the storage unit 18.

[0028] Figure 5 This illustrates a structure where the portable information processing terminal 1 includes a driver 20, but the driver 20 may not be included. The server 2, diagnostic terminal 3, and subject terminal 4 also have... Figure 5 The hardware structures are illustrated, but detailed descriptions of these structures are omitted.

[0029] Figure 6 This is a functional block diagram illustrating an example of the functional structure of the information processing apparatus J according to the first embodiment. The CPU 11 of the portable information processing terminal 1 functions as a control unit 31, a dimming unit 32, and an extraction unit 33. The communication unit 19 communicates with the server 2 via a predetermined communication protocol such as the Internet protocol.

[0030] The storage unit 18 pre-stores the first trained model M1, various thresholds and benchmark values ​​used in the judgment process, etc.

[0031] The output unit 16 has a display 41 for displaying images and a speaker 42 for outputting sound. The input unit 17 has a touch panel 43 for the user to input instructions to the portable information processing terminal 1, a camera 44 for capturing images of the subject's fundus, and a microphone 45 for collecting sound. The touch panel 43 is located on the surface of the display 41 where images are displayed.

[0032] Figure 7 This is a block diagram illustrating a structural example of a camera 44. The camera 44 has an image-capturing unit 51 and a light-emitting unit 61. The image-capturing unit 51 has an image-capturing lens 52 and an image sensor 53. The image sensor 53 is, for example, composed of a CCD or a CMOS sensor.

[0033] The light-emitting unit 61 includes an LED 62 that functions as a lighting unit and a voltage regulation circuit 63. The LED 62 is, for example, an SMD type. The voltage regulation circuit 63, under the control of the dimming unit 32, regulates the DC voltage supplied from a battery (not shown) to the LED 62.

[0034] The control unit 31 controls each unit according to the user-defined action mode. When set to the normal shooting mode, the control unit 31 shoots according to the shooting instructions for still images or moving images. When shooting moving images, in addition to shooting moving images (videos) with multiple frames, it also includes shooting still images repeatedly at predetermined time intervals, with still images forming frames. When shooting moving images, the camera unit 51 shoots multiple still images, for example, at a frame rate of 30 [fps]. The frame rate value is not limited to 30 [fps]; other values ​​can be set and stored in the storage unit 18.

[0035] Furthermore, when set to inspection mode, the control unit 31 causes the camera unit 51 to capture moving images and determines whether an image capturing the frame of the objective lens 92 exists among the multiple captured still image data. If the control unit 31 determines that a still image capturing the frame of the objective lens 92 exists, it transitions to standby mode. When a shooting instruction is input in standby mode, the control unit 31 causes the camera unit 51 to continuously capture moving images instead of still images for a predetermined shooting time. The shooting time is, for example, 5 to 10 seconds. The shooting time is stored in the storage unit 18. Additionally, the control unit 31 divides the moving images acquired from the camera unit 51 in inspection mode into multiple still image data frames and sends the multiple still image data frames to the extraction unit 33.

[0036] That is, the control unit 31 constitutes the shooting instruction unit P1, which ends the shooting after the shooting time has elapsed after the shooting begins. In addition, the control unit 31 also constitutes the still image data acquisition unit P2, which acquires multiple still image data based on the shooting image obtained from the shooting of the subject's fundus after receiving light emitted from the light-emitting unit 61.

[0037] When a shooting instruction is input, the dimming unit 32 acquires an illuminance value representing the brightness of the subject from the image sensor 53 and compares the acquired illuminance with a predetermined illuminance threshold. In normal shooting mode, if the acquired illuminance is less than the illuminance threshold, the dimming unit 32 controls the voltage regulation circuit 63 to set the light emission level to a preset value (default value) in the portable information processing terminal 1 to make the LED 62 emit light. On the other hand, in check mode, even if the acquired illuminance is less than the illuminance threshold, the dimming unit 32 controls the voltage regulation circuit 63 to make the light emission level of the LED 62 lower than the default value of the portable information processing terminal 1.

[0038] Figure 8A It is a graph showing the lengths of various distances when photographing the fundus of a test subject.

[0039] The control of the luminous intensity of LED 62 by the dimming unit 32 will be explained. For light radiated from the light source LED 62 in a certain direction with a solid angle dΩ, let θ be the two-dimensional angle representing the spread of the light. Let KS[lm] be the luminous flux corresponding to the luminous intensity of LED 62 with a solid angle dΩ, and let KD[cd] be the intensity of the light radiated from LED 62 with a solid angle dΩ. In this case, the luminous intensity KD is expressed by the following equation (1) using the luminous flux KS and the angle θ. KD=KS / {2π[1-cos(θ / 2)]}…(1)

[0040] Furthermore, let the distance from the light source LED62 to the subject be L, and let the illuminance SD[lux] of the subject be expressed by the luminous intensity KD and the distance L using the following formula (2). As can be seen from formula (2), when KD is the same value, the longer the distance L, the smaller the illuminance SD. SD=KD / L 2 …(2)

[0041] like Figure 8A As shown, since distance L3 is between 20 and 77 mm, it is between 0.020 and 0.077 m. Distance L2 is, for example, 0.15 m. Distance L1 is distance L2 plus the diameter of the eyeball (approximately 2.5 cm), therefore it is approximately 0.175 m. Angle θ1 is θ1 = (0.005 / 0.15) × (180° / π) = 1.9°.

[0042] Figure 8B This is an explanatory diagram illustrating how to photograph a subject several meters away using the light-emitting unit 61. Figure 8B The distance L4 shown is, for example, 1~2 [m]. Considering the user in typical shooting modes, such as... Figure 8B As shown, the subject is photographed at a distance L4 from the portable information processing terminal 1. In this case, the dimming unit 32 sets the luminous intensity to a default value to make the light-emitting unit 61 emit light when the acquired illuminance is less than the illuminance threshold. The default value is the maximum luminous intensity of the LED 62. When the luminous flux KS of the LED 62 in the light-emitting unit 61 can be adjusted in the range of 40~60 [lm], the default value is 60 [lm].

[0043] Considering Figure 8A In fundus photography, with Figure 8B The same situation applies to the illuminance SD of the subject when the light-emitting part 61 emits light. Figure 8B In this equation, let the illuminance at distance L2 be SD1, and the illuminance at distance L4 be SD2. The luminous intensity KD of θ1 is the same at both distances L2 and L4. According to equation (2), SD1 = KD / L2 2 =KD / (0.15) 2 SD2=KD / L42 =KD / 2 2 Therefore, it is KD / 1 2 The ratio of illuminance SD1 to SD2 is calculated, with SD1 / SD2 values ​​ranging from 1 / 0.0225 to 4 / 0.0225, representing a multiplier of 44 to 177. That is, if the light emission level of the light-emitting unit 61 is used to illuminate a subject several meters away in darkness to photograph the fundus, excessive light will be directed at the eye and fundus. Therefore, in inspection mode, the dimming unit 32 sets the light emission level of the LED 62 of the light-emitting unit 61 to a value lower than the default value of the portable information processing terminal 1.

[0044] Generally, the exposure is calculated based on the integral of the intensity of illuminance SD and the exposure time. The intensity of illuminance SD is determined by the distance L and the aperture value in equation (2), but the aperture value is usually fixed in portable information processing terminals 1 such as smartphones. Therefore, in portable information processing terminals 1, the exposure of the subject's fundus can be considered to be determined by the exposure time and illuminance SD. However, as referred to Figure 8A and 8B As explained, when the illuminance SD value is too high, although the exposure time needs to be shortened, the shutter speed is limited. Therefore, in order to suppress the subject's reflexive eyeball 70 and eyelid movements during shooting and to obtain appropriate exposure to the fundus, the light emission of the light-emitting unit 61 is adjusted by the dimming unit 32 instead of adjusting the exposure time.

[0045] Furthermore, the dimming unit 32 can also change the reference value of the light emission amount according to the type of portable information processing terminal 1. For example, the portable information processing terminal 1 has two types, A and B. In type A, the luminous flux KS of the light-emitting unit 61 can be selected from three levels: 40, 50, and 60 [lm] within the range of 40 to 60 [lm]. In type B, the luminous flux KS of the light-emitting unit 61 can be selected from four levels: 50, 60, 70, and 80 [lm] within the range of 50 to 80 [lm]. When the portable information processing terminal 1 is type A, the dimming unit 32 sets the light emission amount in the inspection mode to the middle level of 50 [lm] among the three levels. On the other hand, when the portable information processing terminal 1 is type B, the dimming unit 32 sets the light emission amount in the inspection mode to the lowest level of 50 [lm] among the four levels. In this way, even if the default value of the light emission amount of the light-emitting unit 61 is set to the maximum value of the light-emitting unit 61, and the default value varies depending on the type of portable information processing terminal 1, it can be adjusted to an appropriate light emission amount.

[0046] After receiving multiple still image data from the camera unit 51, the extraction unit 33 extracts extracted image data that meets predetermined criteria based on the first trained model M1. The extracted image data is the still image data extracted by the extraction unit 33. Specifically, based on the first trained model M1, the extraction unit 33 determines that still image data in the fundus image where at least one of the optic nerve head 84, retinal vessels 85, and macula 82 is clearly imaged is considered extracted image data that meets predetermined conditions. The number of extracted image data can be one or more. The extraction unit 33 sends one or more extracted image data to the server 2 via the communication unit 19. That is, for each of the multiple still image data, the extraction unit 33 extracts extracted image data showing the fundus. More specifically, the extraction unit 33 inputs the static image data acquired by the static image data acquisition unit P2 into the first trained model M1 to obtain a score, and extracts image data based on the score. The first trained model M1 is generated by training with a first training image showing at least one of the optic nerve head 84, retinal vessels 85, and macula 82 of the subject. For example, when the extraction unit 33 inputs the static image data into the first trained model M1 and obtains a score higher than a threshold, it extracts the static image data as the extracted image data. The threshold can be adjusted empirically or experimentally to obtain appropriate extracted image data.

[0047] The first trained model M1 is trained, for example, in the following manner: for more than 4,000 sample images, based on more than 15,000 labeled data, it is trained with a sensitivity of more than 97% using first training images in which at least one of the optic nerve head 84, retinal vessels 85 and macula 82 in the fundus image is clearly imaged, so that the model can determine whether the static image data of the fundus is a quality image that can be used for diagnosis, that is, at least one of the optic nerve head 84, retinal vessels 85 and macula 82 is clearly imaged.

[0048] Next, the processing of the fundus captured by the portable information processing terminal 1 will be explained. Figure 9 This is a flowchart illustrating an example of fundus imaging processing using the information processing device J according to the embodiment. The user operates the touch panel 43 to set the operating mode of the portable information processing terminal 1 to examination mode.

[0049] In S1, the control unit 31 causes the camera unit 51 to capture a moving image and determines whether there is an image capturing the frame of the objective lens 92 among multiple still image data. When the determination result of S1 is that the control unit 31 determines that there is still image data capturing the frame of the objective lens 92, it determines whether a shooting instruction has been input (S2).

[0050] When the judgment result of S2 is that a shooting instruction has been input, the dimming unit 32 sets the light emission of the LED 62 of the light-emitting unit 61 to a value lower than the default value of the portable information processing terminal 1 (S3). In S4, the control unit 31 causes the imaging unit 51 to continuously capture dynamic images of the fundus for a certain period of time. The imaging unit 51 sends multiple still image data acquired by capturing dynamic images to the extraction unit 33.

[0051] In step S5, when the extraction unit 33 receives multiple static image data from the imaging unit 51, it extracts extraction image data that meets predetermined conditions based on the first trained model M1. Specifically, based on the first trained model M1, the extraction unit 33 determines static image data in the fundus image where at least one of the optic nerve head 84, retinal vessels 85, and macula 82 is clearly imaged as extraction image data that meets the predetermined conditions. The number of extracted image data can be one or more.

[0052] In S6, the extraction unit 33 sends one or more extracted image data to the server 2 via the communication unit 19.

[0053] In addition, in S5, if the extraction unit 33 is unable to extract image data that meets the predetermined conditions, it can notify the user by sound via the speaker 42 that the optical axis of the camera lens 52 is misaligned with the optical axis of the objective lens 92.

[0054] Furthermore, in S5, the extraction unit 33 can also perform the process of cropping the image within the objective lens 92 frame from each of the multiple still image data. At this time, the range of the object image for extraction is narrowed, the computational load of the extraction process is reduced, and the processing load of the extraction unit 33 is lessened. As a result, the extraction process of the extraction unit 33 is accelerated. In addition, the capacity of the still image data obtained by cropping the image within the objective lens 92 frame is smaller than the capacity of the still image data without cropping the image within the objective lens 92 frame. As a result, when the extraction unit 33 sends the extracted image data to the server 2 via the communication unit 19 in S6, the communication processing load of the communication unit 19 is also reduced.

[0055] Next, the fundus imaging processing of the information processing device J will be explained. Figure 10 This is a sequence diagram of an example of fundus imaging processing by information processing device J.

[0056] In S11, the portable information processing terminal 1 sends the extracted image data extracted from the fundus image of the test subject to the server 2.

[0057] In S12, when the server 2 receives the extracted image data from the portable information processing terminal 1, it stores the extracted image data and sends it to the diagnostic terminal 3.

[0058] In S13, when the diagnostic terminal 3 receives the extracted image data from the server 2, it displays the fundus image of the extracted image data. The ophthalmologist observes the fundus image displayed on the diagnostic terminal 3 to make a diagnosis and inputs the diagnostic result into the diagnostic terminal 3. In S14, the diagnostic terminal 3 sends the input diagnostic result to the server 2. In S15, when the server 2 receives the diagnostic result from the diagnostic terminal 3, it stores the diagnostic result and sends it to the subject's terminal 4. In S16, when the subject's terminal 4 receives the diagnostic result from the server 2, it displays the diagnostic result.

[0059] In addition, in the implementation, the portable information processing terminal 1 captures a moving image in the inspection mode, but it is also possible to capture a still image instead of a moving image.

[0060] Furthermore, the implementation describes an example where the dimming unit 32 can select three or four levels of light emission, but it is not limited to this. The light emission can also be adjusted using a slider to make the light emission a value lower than the default value.

[0061] Furthermore, while the embodiment describes an example where the light-emitting part 61 is located in the portable information processing terminal 1, the light-emitting part 61 can also be located in the adapter 90. It is preferable that the light-emitting part 61 is located in the portable information processing terminal 1 when the subject requires pupil dilation, but it is also possible to locate the light-emitting part 61 in the adapter 90 when the subject does not require pupil dilation.

[0062] <Advantages of the First Embodiment> According to the information processing device J, when the portable information processing terminal 1 captures images of the subject's fundus, the light emission level of the light-emitting unit 61 of the portable information processing terminal 1 is lower than the default value to suppress the movement of the eyeball 70 and eyelids caused by the reflection reaction during light emission. As a result, changes in the positional relationship between the objective lens 92 and the fundus are suppressed during image capture. Therefore, doctors can reduce the amount of work required to adjust the positional relationship between the camera 44 and the objective lens 92 during image capture, eliminate the influence of hand tremors, and easily obtain high-quality fundus images.

[0063] In existing technologies, the glare from the flash can cause discomfort to test subjects and lead to resistance to fundus examinations. Children, in particular, may remember fundus photography as an unpleasant examination, resisting subsequent examinations and sometimes making the examination impossible. Furthermore, patients with retinal diseases, including corneal disorders, cataracts, and hereditary retinal diseases, are more prone to glare than healthy individuals, experiencing intense discomfort from the flash during imaging, sometimes making the fundus examination feel extremely painful. To address this, the information processing device J uses adjusted light intensity for fundus imaging, ensuring appropriate exposure. Therefore, it reduces glare for the test subjects and avoids discomfort during fundus imaging.

[0064] Furthermore, the dimming unit 32 can adjust the light emission level to a value lower than the default value, corresponding to the type of portable information processing terminal 1. Even if the default value of the light emission level of the light-emitting unit 61 is set to the maximum value of the light-emitting unit 61, and the default value varies depending on the type of portable information processing terminal 1, it can still be adjusted to an appropriate light emission level.

[0065] (Modification 1 of the first embodiment) The portable information processing terminal 1 can take pictures for a certain period of time when the light emission level of the light-emitting unit 61 is lower than the default value. The light-emitting unit 61 emits light for a predetermined period of time at a light emission level adjusted to a value lower than the default value. The camera unit 51 takes pictures of the fundus at a predetermined frame rate within a certain period of time. The extraction unit 33 extracts extracted image data that meets predetermined conditions from multiple static image data of the fundus captured by the camera unit 51.

[0066] According to Modification 1, the brightness of the light illuminating the subject's eye remains constant during fundus photography, thus significantly suppressing eyeball 70 and eyelid movements. Furthermore, since a large amount of static image data can be acquired, the possibility of obtaining high-quality images useful for fundus observation is increased.

[0067] (Modification 2 of the first embodiment) The portable information processing terminal 1 can change the light emission level of the light-emitting unit 61 within a certain period of time, based on a value lower than the default value. The dimming unit 32 changes the light emission level of the light-emitting unit 61 within a predetermined period of time, based on a value lower than the default value. The camera unit 51 captures images of the fundus at a predetermined frame rate within a certain period of time. The extraction unit 33 extracts extracted image data that meets predetermined conditions from multiple still image data of the fundus captured by the camera unit 51.

[0068] Here, as a method for changing the amount of light emitted, two possibilities can be considered: gradually increasing the amount from a reference value and gradually decreasing the amount from a reference value. For example, during fundus imaging, the dimming unit 32 gradually increases the amount of light emitted by the light-emitting unit 61 from a reference value. In this case, when the subject has severe refractive media opacity due to conditions such as cataracts, it is easier to detect abnormalities during fundus observation.

[0069] For example, when the subject complains of photophobia, the doctor can operate the touch panel 43 to input an instruction to reduce the light emission of the light-emitting unit 61. In this case, the dimming unit 32 gradually reduces the light emission of the light-emitting unit 61 from a reference value during fundus imaging. As a result, it is possible to suppress the subject's upward rolling of the eyeball 70 or facial movement due to reflexive eye closure or aversion, and to obtain a good fundus image.

[0070] According to Modification 2, the amount of light emitted by the light-emitting unit 61 onto the subject's eye can be varied (increased or decreased) based on a light emission level lower than the default light emission level of the portable information processing terminal 1, and a large amount of still image data can be acquired during this period. Therefore, the possibility of obtaining appropriately exposed images useful for fundus observation is increased.

[0071] In other words, the information processing device J using the present invention can be implemented in a variety of ways having the following structure.

[0072] That is, the information processing device J includes: a shooting unit (e.g., a camera unit 51) that captures the fundus of the subject when a shooting instruction is input; a light-emitting unit (e.g., a light-emitting unit 61) that emits light when the fundus is captured by the shooting unit; and a dimming unit (e.g., a dimming unit 32) that adjusts the light emission level set in the light-emitting unit to a value lower than a preset value when a shooting instruction is input.

[0073] Therefore, when the information processing device J takes a picture of the subject's fundus, by making the light emission of the light-emitting part 61 of the information processing device J lower than the default value, it is possible to suppress the movement of the subject's eyeball 70 and eyelids caused by the reflection reaction during light emission, and it is easier to obtain high-quality fundus image data.

[0074] Alternatively, it may further include an extraction unit (e.g., extraction section 33) that extracts extracted image data of the fundus images that meet predetermined conditions from multiple static image data of the fundus captured by the shooting unit at predetermined time intervals, and the light emission unit emits light at a predetermined amount of light emission that is adjusted to a low value for a predetermined period of time, and the shooting unit captures images of the fundus at time intervals within a certain period of time.

[0075] Therefore, the brightness of the light shining into the subject's eye remains unchanged during fundus photography, thus significantly suppressing eyeball and eyelid movements.

[0076] Alternatively, it may further include an extraction unit that extracts image data of the fundus images that meet predetermined conditions from multiple static image data of the fundus captured by the shooting unit at predetermined time intervals, and a dimming unit that adjusts the amount of light emitted by the light-emitting unit at a lower value as a reference for varying the amount of light emitted within a predetermined time period, and the shooting unit captures images of the fundus at time intervals within a certain time period.

[0077] Therefore, the amount of light emitted by the light-emitting unit 61 to the subject's eye can be varied based on a light emission level lower than the default light emission level of the information processing device J, while acquiring a large amount of still image data during this period. Thus, the possibility of obtaining appropriately exposed images useful for fundus observation is increased.

[0078] Alternatively, the preset value can be the maximum value of the light emission in the light-emitting unit.

[0079] As a result, the amount of light emitted by the light-emitting unit becomes less than the amount of light emitted when photographing a subject several meters away in the dark, thus suppressing the movement of the subject's eyeballs 70 and eyelids caused by the reflection reaction during light emission.

[0080] Furthermore, the control method is executed by an information processing device J having an imaging unit, a light-emitting unit, and a communication unit, and includes: a dimming process, adjusting the light emission level set in the light-emitting unit to a value lower than a preset value when an imaging instruction is input; an imaging process, causing the imaging unit to capture the fundus of the subject when an imaging instruction is input; and a light emission process, causing the light-emitting unit to emit light when the fundus is captured by the imaging unit.

[0081] Furthermore, the imaging method utilizes an information processing device J having an imaging unit, a light-emitting unit, and a communication unit to image the fundus, including: a dimming process, adjusting the light emission level set in the light-emitting unit to a value lower than a preset value when an imaging instruction is input; an imaging process, causing the imaging unit to image the fundus of the subject when an imaging instruction is input; and a light emission process, causing the light-emitting unit to emit light when the fundus is imaged through the imaging unit.

[0082] Furthermore, the information processing program causes the computer of the information processing device J, which has an imaging unit, a light-emitting unit, and a communication unit, to perform the following steps: a dimming step, in which the light emission level set in the light-emitting unit is adjusted to a value lower than a preset value when an imaging instruction is input; an imaging step, in which the imaging unit is used to image the fundus of the subject when an imaging instruction is input; and a light emission step, in which the light-emitting unit emits light when the fundus is imaged through the imaging unit.

[0083] (Second Implementation) The information processing device J of the second embodiment captures dynamic images of the fundus of the subject, and obtains a portion of static image data from the dynamic image data as images of high quality suitable for fundus observation. In the description of this embodiment, descriptions of structures that are the same as those in other embodiments and modifications are omitted.

[0084] The information processing device J includes: an imaging unit (e.g., camera unit 51) that captures images of the fundus of a subject at predetermined time intervals when an imaging instruction is input; an extraction unit (e.g., extraction unit 33) that extracts extractable image data that meets predetermined conditions from multiple static image data of the fundus captured at time intervals by the imaging unit; and a communication unit (e.g., communication unit 19) that transmits the extractable image data that meets the conditions extracted by the extraction unit to an external device. Thus, multiple static image data of the subject's fundus can be obtained, thereby increasing the possibility of extracting high-quality extractable image data from multiple static image data.

[0085] Alternatively, it may further include an output unit (e.g., output section 16) that notifies the subject of the remaining time required for shooting when the shooting unit begins shooting. Thus, by informing the subject of the remaining shooting time, the subject is able to make an effort to keep their eyeballs 70 and eyelids still for the remaining time. Therefore, the possibility of obtaining a high-quality still image is increased.

[0086] Alternatively, the shooting unit can terminate the shooting process after the predetermined shooting time has elapsed since the start of shooting, or after the extraction unit has extracted the image data that meets the conditions. This reduces the processing load of extracting the image data that meets the conditions from multiple still image datasets.

[0087] Alternatively, the extraction unit may, based on a pre-stored first trained model M1, determine static image data in the fundus image where at least one of the optic nerve head 84, retinal vessels 85, and macula 82 is identified as clearly imaged, and classify this as image data that meets predetermined conditions. Thus, an image of quality suitable for fundus observation is obtained.

[0088] Furthermore, the control method is a control method executed by an information processing device J having an imaging unit and a communication unit, including: an imaging step, which captures the fundus of the subject at predetermined time intervals when an imaging instruction is input; an extraction step, which extracts extracted image data of the captured fundus images that meet predetermined conditions from multiple static image data of the fundus captured at time intervals; and a communication step, which sends the extracted extracted image data that meets the conditions to an external device.

[0089] Furthermore, the information processing program causes the computer of the information processing device J, which has a shooting unit and a communication unit, to perform the following steps: a shooting step, in which the fundus of the subject is photographed at predetermined time intervals when a shooting instruction is input; an extraction step, in which the extracted fundus images satisfy predetermined conditions are extracted from multiple static image data of the fundus photographed at time intervals; and a communication step, in which the extracted image data satisfying the conditions is sent to the outside. <Advantages of the Second Embodiment> According to the information processing device J, multiple still image data of the subject's fundus can be obtained, thus increasing the possibility of extracting high-quality still image data from multiple still image data. Previously, if a user wanted to operate the portable information processing terminal 150 to capture multiple still images, multiple shooting instructions needed to be entered. Each time the user entered a shooting instruction into the portable information processing terminal 150, there was a possibility that the positional relationship between the camera 44, the objective lens 92, and the subject's eye might be misaligned. In contrast, according to the information processing device J, the user only needs to enter a shooting instruction once into the portable information processing terminal 150, and the portable information processing terminal 1 can acquire multiple still images of the fundus. Therefore, the user can focus on maintaining the proper positional relationship between the camera 44 and the objective lens 92 during shooting, and can obtain a set of high-quality still image data.

[0090] Furthermore, according to the information processing device J, it is possible to extract clear image data without hand tremors from multiple static image data, thus eliminating the influence of hand tremors.

[0091] Alternatively, the portable information processing terminal 1 can send the dynamic image data to an external server 2, where the server 2 can extract the extracted image data that meets the criteria from multiple static image data sets. However, the communication environment of the network 100 in developing countries or remote areas may be poor. When the communication environment of the network 100 is poor, the portable information processing terminal 1 may sometimes be unable to send the dynamic image data to the server 2. In this case, the portable information processing terminal 1 will send the extracted image data extracted from the dynamic image data to the external server 2. Since the portable information processing terminal 1 sends the extracted image data, which has a smaller data volume than the dynamic image data, to the server 2, the possibility of being able to send data is increased even in poor communication environments.

[0092] (Modification 1 of the second embodiment) The portable information processing terminal 1 can notify the remaining shooting time during the capture of a moving image. The output unit 16 notifies the remaining shooting time required when the camera unit 51 starts shooting in inspection mode. The control unit 31, for example, causes the speaker 42 to output the remaining time (= shooting time - elapsed time since shooting began) audibly. According to Modification 1, by notifying the subject of the remaining shooting time, the subject is able to make an effort to keep their eyes 70 and eyelids still for the remaining time. Therefore, the possibility of obtaining a high-quality still image is increased.

[0093] (Modification 2 of the second embodiment) The portable information processing terminal 1 can end the shooting when the shooting time for capturing dynamic images reaches the shooting time limit, or when extractable image data that meets the conditions is extracted during the shooting time. In inspection mode, the control unit 31 causes the camera unit 51 to end the shooting when the elapsed time after the start of shooting reaches the shooting time limit, or when the extraction unit 33 extracts extractable image data that meets the conditions. According to Modification 2, the processing load of extracting extractable image data that meets the conditions from multiple static image data can be reduced. The processing of extracting images of suitable quality for fundus observation from multiple static images is performed on the portable information processing terminal 1, which has a lower computing power than the server 2. Therefore, the effect of reducing the processing load is very important for the portable information processing terminal 1.

[0094] (Third Implementation) The information processing apparatus J according to the third embodiment will be described. In the description of this embodiment, the structure that is the same as that in other embodiments and modifications is omitted.

[0095] Generally, the pupil diameter (d) in a dilated state in low light is approximately 4mm to 8mm. In a constricted state in bright light, the pupil diameter (d) is approximately 2mm to 4mm. When the pupil diameter (d) becomes too small due to constriction, it is difficult to obtain a clear fundus image.

[0096] The information processing device J adjusts the amount of light emitted by the light-emitting unit 61 so that the pupil diameter d does not become too small due to pupil constriction.

[0097] In the CPU 11 of the portable information processing terminal 1, in addition to the dimming unit 32 implemented by the program read from the storage unit 18, a pupil diameter calculation unit 34, a disease inference unit 35, and an inference result output unit 36 ​​are also provided. Figure 6 (The double-dotted line). The storage unit 18 stores the second trained model M2 for use by the disease inference unit 35.

[0098] The dimming unit 32 adjusts the amount of light emitted by the light-emitting unit 61 directed towards the subject's eyes to ensure that the subject's pupil diameter d is not less than a predetermined diameter. More specifically, when the pupil diameter calculation unit 34 calculates that the subject's pupil diameter d is less than the predetermined diameter based on static image data, the dimming unit 32 adjusts the light emitted by the light-emitting unit 61 to reduce the amount of light emitted by the light-emitting unit 61 by a predetermined amount. For example, if the predetermined diameter is 3 mm and the predetermined amount is 1 / 4 of the current light emission, and the eye receives light with a light emission of 50 [lm] emitted by the light-emitting unit 61 and the pupil constricts, and the pupil diameter d becomes less than 3 mm, the dimming unit 32 reduces the light emission by 1 / 4 to 37.5 [lm]. The predetermined diameter is not limited to 3 mm; it can also be other sizes such as 2 mm or 4 mm, and can be adjusted empirically or experimentally to obtain clearer static image data. Furthermore, the predetermined amount is not limited to a quantity expressed as a ratio; it can also be a quantitative value such as 10 [lm], and can be adjusted empirically or experimentally to adjust the light emission in a shorter time.

[0099] The pupil diameter calculation unit 34 calculates the pupil diameter d based on static image data. More specifically, the pupil diameter calculation unit 34 acquires the contour image of the pupil 79 using techniques such as edge detection based on the static image data. Furthermore, the pupil diameter calculation unit 34 reads a reference contour image and a reference contour diameter corresponding to the reference contour image stored in the storage unit 18. The pupil diameter calculation unit 34 derives the ratio between the acquired contour image and the reference contour image through a scaling process, and calculates the pupil diameter d by multiplying the reference contour diameter by the ratio. This method of calculating the pupil diameter d is one example; the pupil diameter calculation unit 34 can also use other methods to calculate the pupil diameter d based on static image data.

[0100] The disease inference unit 35 inputs the extracted image data into a second trained model M2 to infer the subject's fundus disease. The second trained model M2 is generated by training with second training images presenting the subject's fundus disease. More specifically, the disease inference unit 35 obtains inferred values ​​for the fundus disease by inputting the extracted image data into the second trained model M2.

[0101] The second training image is, for example, a fundus image that a doctor has diagnosed or suspected as glaucoma, and is a two-dimensional fundus image taken with or without mydriasis. Preferably, the second training image is a fundus image showing the optic disc 84.

[0102] The inference result output unit 36 ​​sends the extracted image data and the inferred values ​​obtained by the disease inference unit 35 to the server 2. The server 2 sends the extracted image data and inferred values ​​to the diagnostic terminal 3, displaying the inference results based on the extracted image data and inferred values ​​on the screen. The inference results can be divided into multiple levels for hierarchical display, making it easier for doctors and patients to understand.

[0103] Next, we will explain the light intensity adjustment process.

[0104] Figure 11 This is a flowchart illustrating an example of light intensity adjustment processing.

[0105] The user installs the portable information processing terminal 1 onto the adapter 90, so that the objective lens 92 is facing the subject's eye.

[0106] Based on the user's input, the light-emitting unit 61 emits light, and the camera unit 51 begins image capture (S21). Still image data is acquired based on the image captured by the camera unit 51 (S22). Then, the pupil diameter d is calculated based on the still image data (S23).

[0107] When the pupil diameter d is less than the specified diameter (S24: Yes), the amount of light emitted by the light-emitting unit 61 is reduced by a specified amount (S25), and the process returns to S22. On the other hand, when the pupil diameter d is not less than the specified diameter (S24: No), the process proceeds to S26. The process of acquiring static image data, calculating the pupil diameter d based on the static image data, and adjusting the amount of light emitted is repeated until the pupil diameter d is greater than the specified diameter (S22~S25).

[0108] If the shooting time has not elapsed (S26: No), the process returns to S22; if the shooting time has elapsed, the process proceeds to S27 (S26: Yes).

[0109] The static image data is input into the first trained model M1. When the output score is higher than the threshold, it is determined that the fundus is clearly presented in the static image data (S27: Yes), and the static image data is extracted into extracted image data (S28). On the other hand, if it is determined that the fundus is not clearly presented in the static image data (S27: No), the process returns to S22.

[0110] The extracted image data is input into the second trained model M2, and the inferred values ​​of fundus diseases are used to infer the fundus diseases of the subject (S28). The inference results are displayed on the screen of the diagnostic terminal 3 (S29).

[0111] That is, the information processing device J includes: a dimming unit 32 that adjusts the amount of light emitted by the light-emitting unit 61 directed toward the eyes of the subject; a static image data acquisition unit P2 that acquires multiple static image data based on a captured image obtained from the fundus of the subject receiving the light emitted by the light-emitting unit 61; and an extraction unit 33 that extracts extracted image data containing the fundus from each of the multiple static image data.

[0112] The dimming unit 32 adjusts the amount of light emitted by the light-emitting unit 61 directed toward the eyes of the test subject so that the pupil diameter d of the test subject is not less than a specified diameter.

[0113] The pupil diameter calculation unit 34 calculates the pupil diameter d based on static image data, and the dimming unit 32 adjusts the light emitted by the light-emitting unit 61 by a predetermined amount when the pupil diameter d is less than a specified diameter.

[0114] The extraction unit 33 inputs the static image data acquired by the static image data acquisition unit P2 into the first trained model M1 to obtain a score, and extracts the extracted image data based on the score. The first trained model M1 is generated by training with a first training image that presents at least one of the optic nerve head 84, retinal vessels 85 and macula 82 of the subject.

[0115] The disease inference unit 35 inputs the extracted image data into the second trained model M2 to infer the fundus disease of the subject. The second trained model M2 is generated by training with a second training image showing the fundus disease of the subject.

[0116] The imaging instruction unit P1 stopped imaging after the required imaging time had elapsed after the start of imaging the subject's fundus.

[0117] In the control method, the dimming unit 32 adjusts the amount of light emitted by the light-emitting unit 61 facing the eyes of the test subject so that the pupil diameter d of the test subject is not less than a specified diameter; the static image data acquisition unit P2 acquires multiple static image data based on the captured image obtained from the fundus of the test subject receiving the light emitted by the light-emitting unit 61; and the extraction unit 33 extracts the extracted image data showing the fundus from each of the multiple static image data.

[0118] The information processing program causes the computer to execute the code of the dimming unit 32, the code of the still image data acquisition unit P2, and the code of the extraction unit 33. The dimming unit 32 adjusts the amount of light emitted by the light-emitting unit 61 directed toward the subject's eyes so that the subject's pupil diameter d is not less than a specified diameter. The still image data acquisition unit P2 acquires multiple still image data based on the captured image obtained from the subject's fundus that receives the light emitted by the light-emitting unit 61. The extraction unit 33 extracts the extracted image data showing the fundus from each of the multiple still image data.

[0119] <Advantages of the Third Embodiment> Therefore, the information processing device J instructs the light-emitting unit 61 to emit light in an amount corresponding to the pupil diameter d through the dimming unit 32. This can suppress eye position shift, pupil constriction, and eye closure caused by aversion reflection due to receiving a large amount of light, and can also reduce discomfort to the subject. In addition, by ensuring that the pupil diameter d is not less than a specified diameter, it is possible to acquire and extract image data based on clearer static image data suitable for fundus observation.

[0120] Furthermore, according to the information processing device J, by utilizing the first trained model M1, it is possible to extract clear image data without hand tremors from multiple static image data, thus eliminating the influence of hand tremors.

[0121] Furthermore, according to the information processing device J, by utilizing the second trained model M2, it is possible to infer the fundus disease of the subject, and to display the extracted image data and the inference results of the fundus disease to the user, thereby assisting in the diagnosis of fundus diseases.

[0122] According to the information processing device J, fundus images suitable for fundus observation can be easily acquired.

[0123] (Fourth Implementation) The information processing apparatus J of the fourth embodiment will be described. Figure 12 This is an explanatory diagram illustrating the optical system for capturing images of the fundus in the information processing device J. Figure 13 This is a diagram showing the design values ​​of the optical system. In the description of this embodiment, descriptions of structures that are the same as in other embodiments and variations are omitted.

[0124] The information processing device J has multiple optical systems, each with positive optical power, arranged sequentially from the eye side. More specifically, the information processing device J has an objective lens 92 and a subsequent lens group Lg arranged after the objective lens 92 along the optical axis. The subsequent lens group Lg has an exit lens Le that allows light to exit from the adapter 90 and an imaging lens 52 to which the light exiting from the exit lens Le is incident. The objective lens 92 and the exit lens Le are disposed in the lens barrel Br of the adapter 90.

[0125] The light-emitting unit 61 can be positioned near the aperture, at a distance orthogonal to the optical axis from the fundus image to the image sensor 53, with an LED 62 or other illumination device arranged at this location. Alternatively, a reflective component can be arranged at this location, allowing the illumination device to project light onto the reflective component to guide the illumination light into the fundus. Either structure can be used. Figure 12 In the example, it is placed inside the microscope tube Br, but there are no special restrictions.

[0126] The subsequent lens group Lg has an imaging function. Moreover, it is possible to move the entire or part of the subsequent lens group Lg along the front-back direction on the optical axis to achieve focusing from near vision to far vision.

[0127] The subsequent lens group Lg in the optical system can magnify or reduce the image formed by the objective lens 92 and image it on the image sensor 53.

[0128] The objective light height H0 of objective lens 92 can be calculated by equation (3) based on the distance between eyeball 70 and objective lens 92 required by adapter 90, i.e., the distance between eyeball and objective lens WD, and the field of view θ that can accommodate optic nerve head 84 and macula 82. H0=WD×tanθ, tanθ=H0 / WD…(3)

[0129] Next, the focal length of objective lens 92, i.e., the focal length f1 of objective lens 92, is calculated by equation (4). In equation (4), Fno is the F-value of objective lens 92. f1=2×Fno×WD×tanθ, Fno=f1 / (2×H0)…(4)

[0130] In addition, the height He of the object on the retina, including the optic disc 84 and the macula 82, is calculated by Equation (5) based on the focal length of the eyeball 70, namely the focal length fe, and the field of view angle θ. Generally, it is known that the focal length fe of the eyeball 70 in the emmetropic state is about 17 mm. He = fe × tanθ…(5)

[0131] The image of retina 73 is projected onto the imaging point inside the lens barrel Br of adapter 90. The image height projected into the lens barrel Br of adapter 90, i.e., the image height H1 inside the adapter, is calculated by equation (6). H1=He×f1 / fe…(6)

[0132] The magnification M of the subsequent lens group Lg is calculated by Equation (7) based on the image height H2 of the image projected onto the image sensor 53 set in the camera 44 and the image height H1 inside the adapter. |M|=H2 / H1…(7)

[0133] The combined magnification of the eyeball 70 and the objective lens 92, namely the eyeball objective lens magnification M0, is calculated by Equation (8) based on the eyeball focal length fe and the objective lens focal length f1. M0=f1 / fe…(8)

[0134] The adapter 90 is equipped with a portable information processing terminal 1 and is supported by the user's hand so that it faces the subject's eyes. Therefore, it is designed to be compact and lightweight to prevent hand fatigue. More specifically, the inner diameter of the lens barrel Br of the adapter 90 is less than 100 mm, preferably less than 60 mm. In addition, when the diagonal length of the image sensor 53 is 8.5 mm and the short side length is 5.1 mm, the image height H2 of the image sensor is preferably 1 to 60% of the half field of view of the short side length, i.e., in the range of 0.255 mm to 1.53 mm.

[0135] (A) In order to make the inner diameter of the lens barrel Br less than 100mm and obtain an image sensor image height H2 of 0.255mm, let the distance between the eyepiece objectives WD be 35mm, the field of view θ be 35°, Fno be 1.2, and the focal length of the eyeball be 17mm. From Equation (3), the objective lens light height H0 is 24.51mm. From Equation (4), the objective lens focal length f1 is 58.82mm. From Equation (5), the object height He on the retina is -11.90mm. From Equation (6), the image height H1 inside the adapter is -41.184mm. From Equation (7), the magnification M of the subsequent lens group is |0.255mm / -41.184mm|=0.006. Further, from Equation (8), the magnification M0 of the eyepiece objective is 58.82mm / 17mm=3.460.

[0136] (B) In order to make the inner diameter of the lens barrel Br less than 60mm and obtain an image sensor image height H2 of 0.255mm, let the distance between the eyepiece objectives WD be 25mm, the field of view θ be 35°, Fno be 1.2, and the focal length of the eyeball be 17mm. From Equation (3), the objective light height H0 is 17.51mm. From Equation (4), the objective focal length f1 is 42.01mm. From Equation (5), the object height He on the retina is -11.90mm. From Equation (6), the image height H1 inside the adapter is -29.417mm. From Equation (7), the magnification M of the subsequent lens group used to obtain the image sensor image height H2 of 0.255mm is |0.255mm / -29.417mm|=0.009. Further, from Equation (8), the magnification M0 of the eyepiece objective is 42.01mm / 17mm=2.471.

[0137] (C) To obtain an image sensor image height H2 of 1.53 mm, assuming the distance between eyepiece objectives WD is 19 mm, the field of view θ is 15°, Fno is 2.0, and the eyepiece focal length fe is 17 mm, the objective lens height H0 is 5.09 mm according to equation (3), the objective lens focal length f1 is 20.36 mm according to equation (4), the object height He on the retina is -4.56 mm according to equation (5), and the image height H1 inside the adapter is -5.457 mm according to equation (6). According to equation (7), the magnification M of the subsequent lens group used to obtain the image sensor image height H2 of 1.53 mm is |1.53 mm / -5.457 mm| = 0.280. Furthermore, the magnification M0 of the eyepiece objective lens is 20.36 mm / 17 mm = 1.198 according to equation (8).

[0138] (D) To obtain an image sensor image height H2 of 1.53 mm, assuming the distance between eyepiece objectives WD is 20 mm, the field of view θ is 15°, Fno is 1.2, and the eyepiece focal length fe is 17 mm, the objective light height H0 is 5.36 mm according to equation (3), the objective lens focal length f1 is 12.86 mm according to equation (4), the object height He on the retina is -4.56 mm according to equation (5), and the image height H1 inside the adapter is -3.446 mm according to equation (6). According to equation (7), the magnification M of the subsequent lens group used to obtain the image sensor image height H2 of 1.53 mm is |1.53 mm / -3.446 mm| = 0.444. Furthermore, the magnification M0 of the eyepiece objective is 12.86 mm / 17 mm = 0.757 according to equation (8).

[0139] The lower limit of the magnification M of the subsequent lens group in the information processing device J is 0.006 as shown in (A), and the upper limit is 0.444 as shown in (C). Furthermore, the lower limit is preferably 0.009 as shown in (B), and the upper limit is preferably 0.280 as shown in (D). The calculation here is based on the diagonal length of the image sensor 53 being 8.5 mm, but the condition for the magnification M of the subsequent lens group when the diagonal length of the image sensor 53 is A is shown in equation (9), and preferably in equation (10). 0.006<|M|×8.5 / A<0.444…(9) 0.009<|M|×8.5 / A<0.280…(10)

[0140] The magnification M0 of the eye objective lens is calculated according to M0=f1 / fe. Since the lower limit of the objective lens focal length f1 is 12.86mm and the upper limit is 58.82mm, and the focal length of the eye is 17mm, the condition for the magnification M0 of the eye objective lens is as shown in equation (11), with a lower limit of 0.757 as shown in (D) and an upper limit of 3.460 as shown in (A). Preferably, the lower limit of the objective lens focal length f1 is 20.36mm and the upper limit is 42.01mm, so the condition for the magnification M0 of the eye objective lens is as shown in equation (12), with a lower limit of 1.198 as shown in (C) and an upper limit of 2.471 as shown in (A). 0.757<|M0|<3.460…(11) 1.198<|M0|<2.471…(12)

[0141] That is, the adapter 90 holds the portable information processing terminals 1 and 150 so that the objective lens 92 and the camera 44 are positioned relative to the subject's eyes at a specified shooting position.

[0142] In addition, the information processing device J has an optical system. When the magnification of the subsequent lens group Lg arranged after the objective lens 92 in the optical axis direction is M and the diagonal length of the image sensor 53 set in the camera 44 is A, the optical system satisfies the condition 0.006 < |M|×A / 8.5 < 0.444.

[0143] The information processing device J has an optical system. When the combined magnification of the eyeball 70 and the objective lens 92 along the optical axis is M0, the optical system satisfies the condition shown in 1.198 < |M0| < 2.471.

[0144] In addition, the information processing device J has an optical system that satisfies the condition 12.86 mm < objective lens focal length f1 < 58.82 mm.

[0145] <Advantages of the Fourth Embodiment> Therefore, the information processing device J can reduce the diameter of the objective lens 92 and the subsequent lens group Lg, making them compact and lightweight so that the user's hand will not be fatigued due to weight and to suppress user hand tremors. In addition, the information processing device J can increase the depth of focus of the optical system, making it easier to focus and acquire high-quality images. Furthermore, the information processing device J can suppress the amount of light refraction on the lens surface, thus suppressing the generation of aberrations and easily obtaining the necessary and sufficient imaging performance.

[0146] (other) Furthermore, the embodiments and variations illustrate an example where objective lens 92 is a single piece, but it is not limited to a single piece; it can also be multiple pieces.

[0147] Furthermore, the embodiments and modifications illustrate an example of using the information processing device J for fundus observation in glaucoma. However, the information processing device J is not limited to glaucoma; it can also be used for fundus diseases such as optic disc cupping, retinal nerve fiber layer defects, diabetes, and side effects of anticancer drugs. In this case, the first trained model M1 and the second trained model M2 are trained using the first training image and the second training image corresponding to the fundus disease.

[0148] Furthermore, the embodiments and variations illustrate an example where the disease inference unit 35 inputs extracted image data into the second trained model M2 to infer fundus diseases, but it is not limited to this; static image data can also be input to infer fundus diseases.

[0149] Furthermore, the aforementioned series of processes can be executed using either hardware or software. In other words, the functional structure described above is merely illustrative and not particularly limited. That is, as long as the information processing device J possesses the capability to execute the aforementioned series of processes as a whole, the type of functional module used to achieve this function is not particularly limited to the example described above. The location of the functional module is also not particularly limited and can be arbitrary. For example, the functional modules of the portable information processing terminal 150 can be transferred to other devices. Conversely, the functional modules of other devices can be transferred to the server 2, etc. In addition, a functional module can be composed solely of hardware, solely of software, or a combination thereof.

[0150] When software performs a series of processes, the programs constituting the software are installed on the computer from the network 100, recording media, etc. The computer can be a computer embedded in dedicated hardware. Alternatively, the computer can be a computer capable of performing various functions by installing various programs, such as a server 2 and other general-purpose personal computers.

[0151] The recording medium containing such a program can be either a removable medium 21 (not shown) that is distributed independently of the device body for the purpose of providing the program to users or the like, or a recording medium that is provided to users or the like in a state of being pre-assembled with the device body. The program can be distributed via network 100, so the recording medium can also be mounted on or accessible by a computer connected to or connected to network 100.

[0152] Furthermore, in this specification, the steps describing the program recorded on the recording medium naturally include processing performed sequentially, but also include processing that does not need to be performed sequentially but is executed in parallel or independently. Additionally, in this specification, the term "system" refers to an integrated device composed of multiple devices, multiple units, etc.

[0153] The above describes one embodiment of the present invention, but the present invention is not limited to the above embodiment. Modifications and improvements within the scope of achieving the purpose of the present invention are also included in the present invention.

[0154] <Advantages of the Invention> According to the present invention, the following effects are achieved.

[0155] (A) According to the information processing device J, the movement of the eyeball 70 and eyelid caused by the reflection reaction during light emission can be suppressed. Doctors can reduce the operation required to adjust the positional relationship between the camera 44 and the objective lens 92 during shooting, eliminate the influence of hand tremors, and easily obtain high-quality fundus images.

[0156] (B) According to the information processing device J, it is possible to reduce the glare felt by the test subject and avoid discomfort caused by fundus photography.

[0157] (C) The portable information processing terminal 1 sends extracted image data to the server 2. The data capacity is smaller than that of the dynamic image data, so the possibility of sending data is increased even in poor communication environments.

[0158] (D) By instructing the light-emitting unit 61 to emit light in an amount corresponding to the pupil diameter d through the dimming unit 32, it is possible to suppress the aversion reflection caused by receiving a large amount of light, which leads to eye position shift, pupil constriction, and eye closure. It can also reduce the discomfort caused to the test subject. In addition, by ensuring that the pupil diameter d is not less than the specified diameter, it is possible to acquire and extract image data based on clearer static image data suitable for fundus observation.

[0159] (E) According to the information processing device J, by utilizing the first trained model M1, it is possible to extract clear image data without hand tremors from multiple static image data, thereby eliminating the influence of hand tremors.

[0160] (F) According to the information processing device J, by using the second trained model M2, it is possible to infer the fundus disease of the subject, and to display the extracted image data to the user as well as the inference results of the fundus disease, thereby assisting in the diagnosis of fundus disease.

[0161] (G) The information processing device J can reduce the diameter of the objective lens 92 and the subsequent lens group Lg, making them compact and lightweight so that the user's hand will not be fatigued by the weight and the user's hand tremors will be suppressed.

[0162] (H) The information processing device J can increase the depth of focus of the optical system, making it easier to focus and acquire high-quality images.

[0163] (I) The information processing device J can suppress the amount of light refraction on the lens surface, thus suppressing the generation of aberrations and easily obtaining the necessary and sufficient imaging performance.

[0164] According to the present invention, an information processing apparatus J, a control method, and an information processing program are provided that can easily acquire fundus images suitable for fundus observation. Explanation of reference numerals in the attached figures

[0165] 1…Portable information processing terminal, 2…Server, 3…Diagnostic terminal, 4…Subject terminal, 11…CPU, 18…Storage unit, 31…Control unit, 32…Dimming unit, 33…Extraction unit, 34…Pupil diameter calculation unit, 35…Disease inference unit, 36…Inference result output unit, 51…Camera unit, 52…Camera lens, 53…Image sensor, 61…Light emission unit, 90…Adapter, 92…Objective lens, J…Information processing device, Le…Outgoing lens, Lg…Subsequent lens group, M1…Trained model, M2…Trained model, P1…Shooting instruction unit, P2…Still image data acquisition unit.

Claims

1. An information processing device, characterized in that, include: The dimming unit adjusts the amount of light emitted from the light-emitting part directed toward the eyes of the test subject; The static image data acquisition unit acquires multiple static image data based on a captured image obtained from the fundus of the subject receiving light emitted by the light-emitting unit; and The extraction unit extracts the extracted image data containing the fundus from each of the plurality of static image data.

2. The information processing apparatus as described in claim 1, characterized in that: The dimming unit adjusts the amount of light emitted by the light-emitting unit directed toward the eyes of the test subject, so that the pupil diameter of the test subject is not less than a specified diameter.

3. The information processing apparatus as described in claim 2, characterized in that: It has a pupil diameter calculation unit. The pupil diameter calculation unit calculates the pupil diameter based on the static image data. The dimming unit adjusts the light emitted by the light-emitting unit when the pupil diameter is less than a specified diameter, thereby reducing the amount of light emitted by the light-emitting unit by a specified amount.

4. The information processing apparatus as described in claim 1, characterized in that: The extraction unit inputs the static image data acquired by the static image data acquisition unit into the first trained model to obtain a score, and extracts the extracted image data based on the score. The first trained model is generated by training with a first training image that presents at least one of the optic nerve head, retinal vessels and macula of the subject.

5. The information processing apparatus as described in claim 1, characterized in that: It has a disease inference department. The disease inference unit inputs the extracted image data into a second trained model to infer the subject's fundus disease. The second trained model is generated by training with a second training image showing the fundus disease of the subject.

6. The information processing apparatus as described in claim 1, characterized in that: Equipped with a shooting indicator, After the shooting instruction unit begins shooting the fundus of the subject, it ends the shooting after the shooting time has elapsed.

7. The information processing apparatus as described in claim 1, characterized in that: It has an adapter, the adapter having an objective lens disposed on the side of the subject. A portable information processing terminal including a camera. The adapter holds the portable information processing terminal so that the objective lens and the camera are positioned relative to the subject's eye at a predetermined shooting position.

8. The information processing apparatus as described in claim 7, characterized in that: Equipped with an optical system When the combined magnification of the subsequent lens group arranged after the objective lens along the optical axis is M and the diagonal length of the image sensor set in the camera is A, the optical system satisfies the condition 0.006 < |M|×A / 8.5 < 0.

444.

9. The information processing apparatus as described in claim 7, characterized in that: Equipped with an optical system When the combined magnification of the eyeball and the objective lens along the optical axis is M0, the optical system satisfies the condition shown in 1.198 < |M0| < 2.

471.

10. The information processing apparatus as described in claim 7, characterized in that: Equipped with an optical system When the objective lens has a focal length of f1, the optical system satisfies the condition 12.86mm < f1 < 58.82mm.

11. A control method, characterized in that: The amount of light emitted from the light-emitting part facing the subject's eyes is adjusted by the dimming unit; The static image data acquisition unit acquires multiple static image data based on the captured image obtained from the fundus of the subject receiving the light emitted by the light-emitting unit; and The extraction unit extracts the extracted image data containing the fundus from each of the plurality of static image data.

12. An information processing program, characterized in that: This enables the computer to execute the code for the dimming unit, the code for the still image data acquisition unit, and the code for the extraction unit. The dimming unit adjusts the amount of light emitted from the light-emitting unit directed towards the subject's eyes. The static image data acquisition unit acquires multiple static image data based on images captured from the fundus of the subject receiving light emitted by the light-emitting unit. The extraction unit extracts the extracted image data containing the fundus from each of the plurality of static image data.

Citation Information

Patent Citations

  • Screening method for automated detection of vision-degenerative diseases from color fundus images

    US20190191988A1

  • Modular adapters for mobile ophthalmoscopy

    WO2016154558A1