Medical information processing system and processing device

Abstract

To enable early verification of the quality of biological images generated from OCT data. [Solution] The medical information processing system comprises an OCT device, a processing device, and a display device. The processing device includes a first processing unit that generates a biological image of tissue based on OCT data detected by a light-receiving element, and a second processing unit that outputs image quality information obtained by inferring the quality of the biological image based on the OCT data before the first processing unit completes the generation of the biological image of tissue based on the OCT data.

Description

Technical Field

[0001] The present disclosure relates to a medical information processing system that processes data of a subject's tissue acquired based on the principle of optical coherence tomography (OCT) and displays it on a display device, and a processing device that processes data of a subject's tissue acquired from an OCT device.

Background Art

[0002] An OCT device guides measurement light divided from OCT light to a subject, while guiding reference light to a reference optical system, and acquires OCT data based on an interference signal obtained by combining the measurement light reflected by the subject (for example, the subject's eye) and the reference light. The OCT device is used, for example, when obtaining a tomographic image of a living tissue such as an eyeball or skin.

[0003] The OCT data acquired by the OCT device is calculated by a processing device and converted into image data. Here, when the OCT device acquires OCT data, various factors during imaging affect the quality of the data, and as a result, also affect the quality (image quality) of the generated image data. Therefore, conventionally, the quality of an image generated based on the acquired OCT data has been evaluated (for example, see Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, conventionally, the quality of images generated based on OCT data was evaluated based on image data obtained by processing the OCT data. In recent years, as OCT images have become higher resolution, wider angle, and deeper, the amount of OCT data generated during acquisition tends to increase. As a result, the computation time required to generate image data also increases, and conventional methods of evaluating image quality based on image data have the drawback of requiring a long time to obtain evaluation results.

[0006] A typical objective of this disclosure is to provide a medical information processing system and processing device that can rapidly evaluate the quality of images generated based on OCT data and inform examiners of the image quality at an early stage. [Means for solving the problem]

[0007] A typical embodiment of the medical information processing system provided in this disclosure comprises an OCT device, a processing device, and a display device. The OCT device comprises an OCT light source, a branching optical element that branches the light emitted from the OCT light source into measurement light and reference light, an irradiation optical system that irradiates the subject's tissue with the measurement light branched by the branching optical element, and a photodetector that detects the interference signal between the measurement light reflected by the tissue and the reference light branched by the branching optical element as OCT data. The processing device comprises a first processing unit that generates a biological image of the tissue based on the OCT data detected by the photodetector, and a second processing unit that outputs image quality information obtained by inferring the quality of the biological image based on the OCT data before the first processing unit completes the generation of the biological image of the tissue based on the OCT data. The display device displays confirmation information for the examiner to confirm the quality of the biological image of the tissue based on the image quality information output from the second processing unit. The second processing unit outputs image quality information using a pre-trained model that has been trained by machine learning to output image quality information when OCT data of the subject's tissue is input.

[0008] A typical embodiment of the present disclosure provides a processing device that processes OCT data obtained from an OCT device that images the tissue of a subject. The processing device includes an input unit that acquires OCT data from an OCT device, an inference unit that outputs image quality information obtained by estimating the quality of the biological image based on the OCT data before the generation of a biological image of the tissue from the OCT data acquired by the input unit is completed, and an output unit that outputs the image quality information output by the inference unit to a display device. The inference unit outputs image quality information using a pre-trained model that has been trained by machine learning to output image quality information when it receives the OCT data of the subject's tissue as input.

[0009] Other processing devices provided by typical embodiments of this disclosure process OCT data obtained from an OCT device that images the tissue of a subject. The processing device comprises an input unit, an inference unit, and an output unit. The input unit acquires multiple B-scan data as OCT data for generating multiple B-scan images, which are two-dimensional biological images of the tissue. The inference unit outputs image quality information obtained by estimating the quality of the three-dimensional biological image by inputting input data related to at least some of the B-scan data before the generation of a three-dimensional biological image of the tissue from the multiple B-scan data acquired by the input unit is completed. The output unit outputs the image quality information output by the inference unit to a display device. The inference unit outputs the image quality information using a pre-trained model that has been trained by machine learning to output image quality information when input data is received.

[0010] According to the medical information processing system and processing device described herein, the quality of biological images can be made known to the examiner at an early stage. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing the overall structure of a medical information processing system. [Figure 2] This figure shows the case where measurement light is scanned in a two-dimensional direction against the fundus tissue within the imaging area. [Figure 3] This is a conceptual diagram illustrating how a three-dimensional image is generated by stacking multiple B-scan images in the Y direction. [Figure 4] This is a block diagram showing the functional configuration of a medical information processing system. [Figure 5] This is a conceptual diagram illustrating how multiple A-scans generate a B-scan image. [Figure 6] This is a conceptual diagram illustrating the process of training a mathematical model using a training dataset to construct a trained model. [Figure 7] This is a conceptual diagram showing a trained model that, when A-scan data is input, outputs image quality information for the corresponding A-scan image. [Figure 8] This is a conceptual diagram illustrating how the image quality information of a corresponding B-scan image is calculated based on multiple image quality information obtained by inputting multiple A-scan data that constitute the B-scan data. [Figure 9] This is a conceptual diagram illustrating a trained model that takes multiple A-scan data sets that make up a B-scan data set as input and outputs image quality information for the corresponding B-scan image. [Figure 10] This is a conceptual diagram showing a trained model constructed to output image quality information for corresponding three-dimensional images by taking multiple B-scan data as input. [Figure 11] This is a diagram showing the display area of ​​a monitor. [Figure 12] This is a flowchart showing the overall control process performed by the medical information processing system. [Figure 13] This figure shows a timing chart of the calculation processing performed by the first and second processing units of the embodiment, in comparison with a comparative example. [Figure 14] This flowchart shows the image quality information calculation process performed by the second processing unit. [Figure 15] This figure shows a timing chart of the calculation processing by the first and second processing units in the modified example, compared with the embodiment. [Figure 16]It is a flowchart showing the automatic reshooting process executed by the second processing unit according to the modified example. [Figure 17] In the medical information processing system according to the modified example, it is a conceptual diagram showing how a generated image is complemented by an estimated image. [Figure 18] It is a flowchart showing the process when one processing unit performs the estimation of image quality and the generation of a biological image in the medical information processing system according to the modified example.

Mode for Carrying Out the Invention

[0012] <Summary> The medical information processing system exemplified in the present disclosure includes an OCT device, a processing device, and a display device. The OCT device includes an OCT light source, a branching optical element that branches the light emitted from the OCT light source into measurement light and reference light, an irradiation optical system that irradiates the measurement light branched by the branching optical element onto the tissue of the subject, the measurement light reflected by the tissue, and a light receiving element that detects the interference signal between the measurement light and the reference light branched by the branching optical element as OCT data. The processing device includes a first processing unit and a second processing unit. The first processing unit generates a biological image of the tissue based on the OCT data detected by the light receiving element. The second processing unit outputs image quality information obtained by inferring the quality of the biological image based on the OCT data before the first processing unit completes the generation of the biological image of the tissue based on the OCT data. The display device displays confirmation information for the examiner to confirm the quality of the biological image of the tissue based on the image quality information output from the second processing unit. The second processing unit outputs the image quality information using a learned model that has been previously trained by machine learning to output the image quality information when the OCT data of the tissue of the subject is input.

[0013] Due to the recent improvement in the image quality of OCT devices, the time required to generate a biological image by calculation based on OCT data tends to be longer. Therefore, when evaluating the quality of the generated biological image, it takes even more time to obtain the evaluation result. Thus, if the image quality is evaluated as poor and the subject needs to be re-photographed, the burden on both the examiner and the subject will increase.

[0014] Therefore, in the medical information processing system of the present disclosure, before the generation of a biological image is completed by the first processing unit, the second processing unit outputs image quality information obtained by inferring the quality of the biological image based on the OCT data. Then, the display device displays confirmation information for the examiner to confirm the quality of the biological image based on the image quality information acquired from the second processing unit. Therefore, without waiting for the generation of the biological image, the examiner can confirm the quality of the biological image via the display device. As a result, the examiner can grasp the quality of the biological image generated at an early timing, and if the quality is low, measures such as causing the OCT device to perform re-photography earlier can be taken.

[0015] Here, the confirmation information is information displayed on the monitor based on the image quality information, and may be, for example, information obtained by visualizing the image quality information. Specifically, when the image quality information is a numerical value, an image obtained by graphing the numerical value may be displayed as the confirmation information. In this case, the confirmation information can be said to be visualization information obtained by visualizing the image quality information. Also, the numerical value of the image quality information itself may be displayed on the monitor as the confirmation information. In this case, the confirmation information can be said to be the image quality information itself.

[0016] Here, "before the first processing unit completes the generation of the biological image" means, for example, when generating an A-scan image based on OCT data obtained from an A-scan, the second processing unit acquires image quality information from the OCT data obtained from the A-scan before the first processing unit generates the A-scan image. Similarly, when generating a B-scan image based on OCT data obtained from a B-scan, the second processing unit acquires image quality information from the OCT data (multiple A-scan data) obtained from the B-scan before the first processing unit generates the B-scan image. Furthermore, when generating a three-dimensional image from OCT data obtained from multiple B-scans, the second processing unit acquires image quality information from the OCT data obtained from the multiple B-scans before the first processing unit generates the three-dimensional image.

[0017] Furthermore, as image quality information, values ​​corresponding to general indicator values ​​for evaluating the quality of biological images may be used. For example, such an indicator value is the signal strength index (SSI) of a biological image generated from OCT data. That is, an estimated value of the SSI may be output as image quality information. However, as image quality information, an estimated value of any of the following, such as SQI (SLO Quality Index), SNR, background noise level, and background contrast, or an estimated value combining them, may also be output. In addition, an estimated value of an index representing the examiner's subjective evaluation (e.g., "good," "poor," "average," etc.) may be output as image quality information.

[0018] The processing unit may be configured separately from the OCT device, for example, as a personal computer. Alternatively, the processing unit may be built into the OCT device and configured as an integrated unit.

[0019] Here, the OCT device may include a scanning unit. The scanning unit scans the measurement light irradiated onto the tissue by the irradiation optical system in a two-dimensional direction intersecting the optical axis. OCT data may be obtained by scanning the spot of measurement light in a two-dimensional direction within the measurement area by the scanning unit. However, it is also possible to change the configuration of the OCT device. For example, the irradiation optical system of the OCT device may simultaneously irradiate a two-dimensional region on the tissue of the subject with measurement light. In this case, the photodetector may be a two-dimensional photodetector that detects interference signals in a two-dimensional region on the tissue. In other words, the OCT device may acquire OCT data by the principle of so-called full-field OCT (FF-OCT). Alternatively, the OCT device may simultaneously irradiate measurement light onto an irradiation line extending in a one-dimensional direction in the tissue, and scan the measurement light in a direction intersecting the irradiation line. In this case, the photodetector may be a one-dimensional photodetector (e.g., a line sensor) or a two-dimensional photodetector. In other words, the OCT device may acquire tomographic images by the principle of so-called line-field OCT (LF-OCT).

[0020] Furthermore, a pre-trained model may be updated in real time based on feedback from the examiner. That is, an input device may be provided that can receive the examiner's evaluation information on the quality of the biological image generated by the first processing unit. The pre-trained model may then be trained in real time based on the evaluation information input via the input device. Alternatively, the pre-trained model may be trained (feedback) in real time using an evaluation value of the quality of the generated biological image (for example, the signal intensity (SSI) of the biological image).

[0021] The light-receiving element may detect A-scan data as OCT data for generating an A-scan image, which is a one-dimensional biological image of tissue. The trained model may be pre-trained to output image quality information regarding the quality of the A-scan image generated based on the A-scan data when A-scan data is input.

[0022] Thus, when a trained model is input with A-scan data, it outputs image quality information regarding the quality of the A-scan image. Therefore, image quality information can be obtained for each A-scan image obtained from each A-scan.

[0023] The display device may display confirmation information for multiple A scan data based on multiple image quality information output from the second processing unit.

[0024] In this way, the display device displays confirmation information for multiple A-scan data based on multiple image quality information acquired from the second processing unit. Therefore, the examiner can confirm the image quality for each A-scan data.

[0025] Here, the light-receiving element detects multiple A-scan data as OCT data to generate a B-scan image, which is a two-dimensional biological image. The second processing unit then calculates image quality information corresponding to the B-scan data based on multiple image quality information corresponding to the multiple A-scan data that make up the B-scan data, output from the trained model. The display device then displays confirmation information about the B-scan data based on the image quality information corresponding to the B-scan data output from the second processing unit.

[0026] In this way, the second processing unit calculates image quality information corresponding to the B-scan data from multiple image quality information corresponding to multiple A-scan data that constitute the B-scan data output from the trained model. Therefore, image quality information corresponding to the B-scan data can be obtained without training the trained model to output image quality information for the B-scan data.

[0027] Here, the light-receiving element detects multiple A-scan data as OCT data, which constitute the B-scan data for generating a B-scan image, a two-dimensional biological image of tissue. The trained model may be pre-trained to output image quality information regarding the quality of the B-scan image generated based on the multiple A-scan data when it is input.

[0028] The trained model, upon input of B-scan data, outputs image quality information regarding the quality of the B-scan image. Therefore, it is possible to obtain image quality information about the B-scan image obtained from the B-scan.

[0029] Furthermore, the trained model may be pre-trained to output an estimate of the signal intensity of a biological image generated based on OCT data as image quality information.

[0030] Thus, the trained model is designed to output an estimate of the signal intensity of a biological image as image quality information. Therefore, it becomes possible to appropriately check the quality of the biological image from the estimated signal intensity.

[0031] Furthermore, the display device may display confirmation information for OCT data corresponding to estimated values ​​smaller than a preset threshold in a different manner than the confirmation information for OCT data corresponding to estimated values ​​larger than the threshold.

[0032] In this way, the display device displays confirmation information for OCT data corresponding to estimates smaller than a preset threshold in a different manner than the confirmation information for OCT data corresponding to estimates larger than the threshold. Therefore, it is possible to easily make low-quality OCT data recognizable to the examiner.

[0033] The processing device illustrated in this disclosure processes OCT data obtained from an OCT device that images the tissue of a subject. The processing device comprises an input unit, an inference unit, and an output unit. The input unit acquires OCT data from the OCT device. The inference unit outputs image quality information obtained by estimating the quality of the biological image based on the OCT data before the generation of a biological image of the tissue from the OCT data acquired by the input unit is completed. The output unit outputs the image quality information output by the inference unit to a display device. The inference unit outputs image quality information using a pre-trained model that has been trained by machine learning to output image quality information when OCT data of the subject's tissue is input.

[0034] In the processing apparatus of this disclosure, before the generation of a biological image based on OCT data is completed, the inference unit outputs image quality information obtained by inferring the quality of the biological image based on the OCT data. The output unit then outputs the image quality information output by the inference unit to a display device. Therefore, without waiting for the biological image to be generated, the examiner can check the quality of the biological image via the display device. This allows the examiner to grasp the quality of the generated biological image at an early stage, and if the quality is low, they can take measures such as resuming imaging early.

[0035] The processing unit may be configured separately from the OCT device, for example, as a personal computer. Alternatively, the processing unit may be built into the OCT device and configured as an integrated unit.

[0036] Here, the input unit acquires A-scan data as OCT data from the OCT device to generate A-scan images, which are one-dimensional biological images of tissue. The trained model may be pre-trained to output image quality information regarding the quality of the A-scan image generated based on the A-scan data when A-scan data is input.

[0037] Thus, when a trained model is input with A-scan data, it outputs image quality information regarding the quality of the A-scan image. Therefore, image quality information can be obtained for each A-scan image obtained from each A-scan.

[0038] Here, the input unit acquires multiple A-scan data as OCT data from the OCT device to constitute B-scan data for generating B-scan images, which are two-dimensional biological images of tissue. The inference unit may then calculate image quality information corresponding to the B-scan data based on multiple image quality information corresponding to the multiple A-scan data that constitute the B-scan data, which are output from the trained model.

[0039] Thus, the inference unit calculates the image quality information corresponding to the B-scan data from multiple image quality information corresponding to multiple A-scan data that constitute the B-scan data output from the trained model. Therefore, it is possible to obtain the image quality information corresponding to the B-scan data without training the trained model to output image quality information for the B-scan data.

[0040] Furthermore, when the inference unit calculates the image quality information corresponding to the B scan data from multiple image quality information corresponding to the A scan data, it can calculate the image quality information corresponding to the B scan data using various methods. For example, the average value of multiple image quality information corresponding to multiple A scan data constituting the B scan data may be calculated, and this average value may be used as the image quality information corresponding to the B scan data.

[0041] Furthermore, the input unit acquires multiple A-scan data as OCT data from the OCT device, which constitutes B-scan data for generating B-scan images, which are two-dimensional biological images of tissue. The trained model may be pre-trained to output image quality information regarding the quality of the B-scan image generated based on the multiple A-scan data when it is input.

[0042] The trained model, upon input of B-scan data, outputs image quality information regarding the quality of the B-scan image. Therefore, it is possible to obtain image quality information about the B-scan image obtained from the B-scan.

[0043] Furthermore, the trained model may be pre-trained to output an estimate of the signal intensity of a biological image generated based on OCT data as image quality information.

[0044] Thus, the trained model is designed to output an estimate of the signal intensity of a biological image as image quality information. Therefore, it becomes possible to appropriately verify the quality of the biological image from the estimated signal intensity.

[0045] In this case, if the estimated value of the OCT data output by the inference unit is lower than a preset threshold, the output unit may output a signal instructing the OCT device to acquire the OCT data again.

[0046] Thus, the output unit outputs a signal to the OCT device instructing it to acquire OCT data again if the estimated value is lower than the threshold. Therefore, the OCT device can automatically re-acquire OCT data that has been determined to be of low quality without any instruction from the examiner.

[0047] Furthermore, the trained model may be trained using multiple OCT data points from multiple subjects' tissues, acquired in advance, as input training data, and the signal intensities of multiple biological images generated based on said OCT data points as output training data.

[0048] Thus, the trained model is trained on a dataset in which OCT data is used as input training data and the signal intensity of biological images actually generated based on that OCT data is used as output training data. Therefore, the trained model can be properly trained.

[0049] Other processing devices illustrated in this disclosure process OCT data obtained from an OCT device that images the tissue of a subject. The processing device comprises an input unit, an inference unit, and an output unit. The input unit acquires multiple B-scan data as OCT data for generating multiple B-scan images, which are two-dimensional biological images of the tissue. The inference unit outputs image quality information obtained by estimating the quality of the three-dimensional biological image by inputting input data related to at least some of the B-scan data acquired by the input unit before the generation of a three-dimensional biological image of the tissue from the multiple B-scan data is completed. The output unit outputs the image quality information output by the inference unit to a display device. The inference unit outputs image quality information using a pre-trained model that has been trained by machine learning to output image quality information when input data is received.

[0050] In the processing apparatus of this disclosure, before the generation of a three-dimensional image based on multiple B-scan data is completed, the inference unit outputs image quality information obtained by inferring the quality of the three-dimensional biological image based on input data related to at least some of the B-scan data. The output unit then outputs the image quality information output by the inference unit to a display device. Therefore, without waiting for the generation of the three-dimensional biological image, the examiner can check the quality of the biological image via the display device. This allows the examiner to grasp the quality of the biological image generated at an early stage, and if the quality is low, they can take measures such as resuming imaging early.

[0051] Here, "input data related to at least some of the B-scan data from multiple B-scan data sets" refers to data related to one or more B-scan data sets from all of the multiple B-scan data sets that make up the three-dimensional image. The related data may be one or more B-scan data sets themselves (i.e., OCT data obtained by the B-scan), or one or more B-scan images generated from one or more B-scan data sets. Furthermore, it may be an evaluation value (e.g., SSI) indicating the quality of one or more B-scan images generated from one or more B-scan data sets.

[0052] Therefore, the input data for training the pre-trained model includes one or a combination of B-scan data, B-scan image data, or evaluation values ​​for assessing the quality of B-scan image data. The output data for training the pre-trained model includes evaluation values ​​(e.g., SSI) for assessing the quality of three-dimensional biological images.

[0053] Here, the input data may be a B-scan image generated from at least some of the B-scan data among a plurality of B-scan data.

[0054] In this way, the overall quality of a three-dimensional biological image can be estimated based on a portion of B-scan images generated from a portion of B-scan data. Specifically, for example, after starting the acquisition, a predetermined number of B-scan images may be generated from a predetermined number of B-scan data (e.g., 30). Then, by inputting the generated predetermined number of B-scan images into a trained model, image quality information of the three-dimensional biological image may be output. Alternatively, all B-scans may be completed, and a predetermined number (e.g., 30) of B-scan data may be extracted randomly or according to a predetermined rule from the acquired B-scan data. Then, B-scan images may be generated from the extracted B-scan data, and image quality information of the three-dimensional biological image may be obtained based on the generated B-scan images.

[0055] <Embodiment> The following describes one typical embodiment of the present disclosure. As an example, the medical information processing system 1 of this embodiment can acquire and process OCT data of fundus tissue using the fundus of the eye E under examination as the subject. Based on the acquired OCT data, it is possible to generate three-dimensional tomographic images and two-dimensional tomographic images, etc. However, even when processing OCT data of biological tissue other than the fundus of the eye E under examination (e.g., the anterior segment of the eye E under examination), or of a subject other than the eye E under examination (e.g., skin, digestive organs, brain, blood vessels (including cardiovascular vessels), or teeth, etc.), at least a part of the technology exemplified in this disclosure can be applied. Here, OCT data is data acquired based on the principle of optical coherence tomography (OCT), and is also called RAW data before it is processed as image data.

[0056] In this embodiment, the medical information processing system 1 comprises an OCT device 10 for acquiring OCT data, a control unit (processing device) 30 for processing the acquired OCT data, and a monitor (display device) 37 for displaying the processing results from the control unit 30. The medical information processing system 1 also includes an operation unit (input device) 38. The medical information processing system 1 in this embodiment is equipped with a microphone 36. However, this microphone 36 is not essential, and it is not required to provide a microphone 36. In this embodiment, the control unit 30 is a computer such as a personal computer, configured separately from the OCT device 10. However, the OCT device 10 and the control unit 30 may be configured as an integrated unit. In other words, the OCT device 10 may have the control unit 30 built into it.

[0057] Referring to Figure 1, the schematic configuration of the OCT apparatus 10 of this embodiment will be described. The OCT apparatus 10 comprises an OCT light source 11, a coupler (optical splitter) 12, a measurement optical system 13, a reference optical system 20, a photodetector 22, and a front observation optical system 23.

[0058] The OCT light source 11 emits light (OCT light) for acquiring OCT data. The coupler 12 splits the OCT light emitted from the OCT light source 11 into measurement light and reference light. In addition, the coupler 12 in this embodiment combines and interferes the measurement light reflected by the subject (the fundus of the eye E in this embodiment) with the reference light generated by the reference optical system 20. In other words, the coupler 12 in this embodiment serves as both a branching optical element that splits the OCT light into measurement light and reference light, and a multiplexing optical element that combines the reflected measurement light and the reference light. It is also possible to change the configuration of at least one of the branching optical element and the multiplexing optical element. For example, elements other than the coupler (e.g., a circulator, beam splitter, etc.) may be used.

[0059] The measurement optical system 13 guides the measurement light, split by the coupler 12, to the subject and returns the measurement light reflected by the subject to the coupler 12. The measurement optical system 13 comprises a scanning unit 14, an illumination optical system 16, and a focus adjustment unit 17. The scanning unit 14 is driven by a drive unit 15 to scan (blind) the measurement light in a two-dimensional direction intersecting the optical axis of the measurement light. In this embodiment, two galvanometer mirrors capable of deflecting the measurement light in different directions are used as the scanning unit 14. However, another device that deflects light (e.g., at least one of a polygon mirror, resonant scanner, or acousto-optic element) may be used as the scanning unit 14. The illumination optical system 16 is located downstream of the scanning unit 14 in the optical path (i.e., on the subject side) and irradiates the subject's tissue with the measurement light. The focus adjustment unit 17 adjusts the focus of the measurement light by moving an optical element (e.g., a lens) provided in the illumination optical system 16 in a direction along the optical axis of the measurement light.

[0060] The reference optical system 20 generates reference light and returns it to the coupler 12. In this embodiment, the reference optical system 20 generates reference light by reflecting the reference light split by the coupler 12 using a reflective optical system (e.g., a reference mirror). However, the configuration of the reference optical system 20 can also be changed. For example, the reference optical system 20 may transmit the light incident from the coupler 12 without reflection and return it to the coupler 12. The reference optical system 20 includes an optical path length difference adjustment unit 21 that changes the optical path length difference between the measurement light and the reference light. In this embodiment, the optical path length difference is changed by moving the reference mirror in the optical axis direction. The configuration for changing the optical path length difference may be provided in the optical path of the measurement optical system 13.

[0061] The photodetector 22 detects the interference signal by receiving the interference light between the measurement light and the reference light generated by the coupler 12. In this embodiment, the principle of Fourier-domain OCT is employed. In Fourier-domain OCT, the spectral intensity of the interference light (spectral interference signal) is detected by the photodetector 22. That is, the OCT device 10 acquires the spectral intensity data of the interference light as OCT data (RAW data). The spectral intensity data acquired by the OCT device 10 is Fourier transformed by the control unit 30 (first processing unit 31, described later) to obtain a complex OCT signal. Examples of Fourier-domain OCT include Spectral-domain-OCT (SD-OCT), Swept-source-OCT (SS-OCT), etc. It is also possible to employ, for example, Time-domain-OCT (TD-OCT).

[0062] In this embodiment, SD-OCT is employed. In the case of SD-OCT, for example, a low-coherent light source (broadband light source) is used as the OCT light source 11, and a spectroscopic optical system (spectrometer) that spectrally separates the interference light into each frequency component (each wavelength component) is provided near the photodetector 22 in the optical path of the interference light. In the case of SS-OCT, for example, a wavelength scanning light source (tunable light source) that changes the output wavelength rapidly over time is used as the OCT light source 11. In this case, the OCT light source 11 may include a light source, a fiber ring resonator, and a wavelength-selective filter. Examples of wavelength-selective filters include filters that combine a diffraction grating and a polygon mirror, and filters that use a Fabry-Perot etalon.

[0063] As described above, in this embodiment, OCT data capable of generating a three-dimensional image 60 is acquired by scanning a spot of measurement light within a two-dimensional region by the scanning unit 14. However, it is also possible to change the principle of acquiring the data for the three-dimensional image 60. For example, OCT data capable of generating a three-dimensional image 60 may be acquired by the principle of line-field OCT (hereinafter referred to as "LF-OCT"). In LF-OCT, measurement light is simultaneously irradiated onto an irradiation line extending in a one-dimensional direction in the tissue, and the interference light of the reflected measurement light and the reference light is received by a one-dimensional photodetector (e.g., a line sensor) or a two-dimensional photodetector. Three-dimensional OCT data is acquired by scanning the measurement light in a direction intersecting the irradiation line within a two-dimensional measurement region. Alternatively, data capable of generating a three-dimensional image 60 may be acquired by the principle of full-field OCT. In full-field OCT, measurement light is simultaneously irradiated onto a two-dimensional region on the tissue of the subject, and the interference light is received by a two-dimensional photodetector.

[0064] As shown in Figure 2, the scanning unit 14 scans light (measurement light) within a two-dimensional imaging area 51 in the biological tissue (in the example shown in Figure 2, the fundus tissue 50). More specifically, the OCT device 10 of this embodiment scans the measurement light within the imaging area 51 on a scan line 52 that extends in the X direction perpendicular to the Z direction. A virtual scan to acquire a one-dimensional image (A scan image) extending in the Z direction is called an A scan. As the measurement light is scanned on the scan line 52, the photodetector 22 detects OCT data (hereinafter referred to as A scan data) corresponding to each A scan at multiple positions on the scan line 52. In the example shown in Figure 2, the Z direction is perpendicular to the two-dimensional imaging area 51 (the depth direction of the fundus tissue 50), and the X direction is the direction in which the scan line 52 extends. Based on the A scan data detected by the photodetector 22, a one-dimensional image (A scan image) of the fundus tissue 50 is generated. Furthermore, one B-scan is achieved by scanning the measurement light along the scan line 52. The OCT data detected by the photodetector 22 during this B-scan (i.e., multiple A-scan data) is considered B-scan data. Based on this B-scan data, a two-dimensional image of the fundus tissue 50 (B-scan image 61) is generated.

[0065] Next, the OCT device 10 moves the position of the scan line 52 in the Y direction within the imaging area 51 and repeats the second and subsequent B scans multiple times. The Y direction intersects (perpendicularly in this embodiment) both the Z and X directions. As a result, multiple B scan data are acquired, passing through each of the multiple scan lines 52 and extending in the depth direction of the fundus tissue 50. Furthermore, by arranging the two-dimensional images generated from each B scan data by the first processing unit 31 of the control unit 30 (described later) in the Y direction (a direction intersecting the image area of ​​each two-dimensional image), a three-dimensional image 60 (see Figure 3) in the imaging area 51 can also be generated. In other words, the image data that can be generated by the medical information processing system 1 of this embodiment is image data of a one-dimensional A scan image based on A scan data in the Z direction, which is the depth direction of the biological tissue, a two-dimensional B scan image 61 based on B scan data, and a three-dimensional image 60 based on multiple B scan data.

[0066] The frontal observation optical system 23 is provided to capture a frontal observation image of the subject's biological tissue (in this embodiment, the fundus tissue 50 of the eye E under examination) in real time. In this embodiment, the frontal observation image is a two-dimensional image of the biological tissue viewed from a direction along the optical axis of the OCT measurement light (frontal direction). In this embodiment, a scanning laser ophthalmoscope (SLO) is used as the frontal observation optical system 23. However, the configuration of the frontal observation optical system 23 may also be other configurations (for example, an infrared camera that captures a frontal image by irradiating a two-dimensional imaging range with infrared light in one step). The OCT device 10 in this embodiment is capable of performing tracking, which makes the scanning position of the OCT light follow the movement of the eye under examination, based on the frontal observation image captured by the frontal observation optical system 23.

[0067] The control unit 30 controls various aspects of the OCT device 10 and performs various processes on the OCT data acquired by the OCT device 10. The control unit 30 comprises a first processing unit 31, a second processing unit 32, a ROM 33, and a non-volatile memory (NVM) 34. The processing procedures of the control unit 30 are usually implemented by software (computer program code), and this software is recorded on a recording medium such as the NVM 34. However, some or all of the processing may be implemented by hardware (dedicated circuitry).

[0068] Herein, in this disclosure, the term “processor” means one or more hardware processors configured to execute program code contained in a program (i.e., one or more instructions of a program). In other words, “processor” is a hardware device capable of executing one or more programmed processes. For example, “processor” may be a general-purpose or application-specific processor and may be, but is not limited to, a CPU, microprocessor, GPU, and DFP (Data Flow Processor).

[0069] In this disclosure, the term “memory” refers to one or more hardware memories that are non-transitional tangible recording media configured to record computer program code and / or data in a manner accessible from a processor. “Memory” can be implemented by memory technologies such as SRAM, SDRAM, non-volatile / flash type memory, or other types of memory. The computer program code that constitutes the program is recorded on the NVM34 and executed by the processor, thereby enabling the control unit 30 to implement various functions.

[0070] ROM33 stores various programs and initial values. NVM34 is a non-transient storage medium that retains its contents even when the power supply is cut off. Programs for executing control processing (see Figure 12) and image quality information calculation processing (see Figure 14) performed by the medical information processing system 1, which will be described later, are stored in NVM34.

[0071] (Regarding the first processing unit 31) The first processing unit 31 generates medical images (ophthalmic images in this embodiment) based on OCT data acquired by the OCT device 10. The first processing unit 31 is also configured to analyze the generated medical images. Figure 4 is a block diagram showing the functional parts of the control unit 30, and the first processing unit 31 mainly consists of an analysis unit 31A and a calculation unit 31B. The calculation unit 31B is composed of, for example, a GPU (Graphic Processing Unit). The calculation unit 31B performs calculations on the OCT data, for example, a Discrete Fourier Transform, to generate medical image data (hereinafter simply referred to as image data).

[0072] Specifically, the calculation unit 31B generates image data of the A-scan image based on the A-scan data acquired by the light-receiving element 22. Also, as shown in Figure 5, the calculation unit 31B generates image data of the B-scan image 61 based on the multiple A-scan data that constitute the B-scan data. Furthermore, as shown in Figure 3, the calculation unit 31B generates image data of the three-dimensional image 60 by superimposing the multiple B-scan images 61 generated based on the multiple B-scan data in the Y direction.

[0073] The analysis unit 31A is composed of, for example, a CPU (Central Processing Unit). The analysis unit 31A analyzes the image data generated by the calculation unit 31B and outputs the analysis results along with the image data to the monitor 37. Specifically, for example, it analyzes the image data of a two-dimensional image generated by the calculation unit 31B to determine whether or not there are any abnormalities such as diseases in the two-dimensional image. The analysis unit 31 may also analyze the image data generated by the calculation unit 31B to detect the position of at least one of the layers and boundaries depicted in the image.

[0074] (Regarding the second processing unit 32) As shown in Figure 4, the second processing unit 32 consists of an input unit 32A, an inference unit 32B, and an output unit 32C. In this embodiment, the second processing unit 32 further includes a determination unit 32D. OCT data is input to the input unit 32A from the OCT device 10. The OCT data acquired by the input unit 32A is sent to the inference unit 32B. The inference unit 32B is composed of, for example, a CPU, and calculates information regarding the quality of the OCT data through inference. More specifically, the inference unit 32B calculates image quality information, which is information regarding the quality of the OCT data, by executing a program that implements a pre-trained algorithm stored in the NVM 34.

[0075] Here, image quality information refers to an estimated evaluation value of the quality (image quality) of a biological image generated based on OCT data acquired from the OCT device 10. In other words, the image quality information output by the inference unit 32B can be said to be a value calculated by estimation of the quality of a biological image generated from OCT data without generating a biological image. As image quality information, values ​​corresponding to general index values ​​for evaluating the quality of a biological image may be used. For example, one such index value is the signal strength index (SSI) of a biological image generated from OCT data. That is, the inference unit 32B may output an estimated value of SSI (hereinafter referred to as estimated SSI) as image quality information. However, as image quality information, it may output an estimated value of any of the following: SQI (SLO Quality Index), SNR, background noise level, and background contrast, or an estimated value combining them. Furthermore, as image quality information, it may output an estimated value of an index representing the subjective evaluation by the examiner (for example, "good," "bad," "medium," etc.).

[0076] (Regarding the pre-trained model 40) The pre-trained model 40 used by the inference unit 32B is constructed by training a mathematical model 42 using a machine learning algorithm. Commonly known machine learning algorithms include neural networks, random forests, boosting, and support vector machines (SVMs).

[0077] Neural networks are techniques that mimic the behavior of biological nerve cell networks. Examples of neural networks include feedforward neural networks, RBF networks (radiating basis function networks), spiking neural networks, convolutional neural networks, recurrent neural networks (recurrent neural networks, feedback neural networks, etc.), and probabilistic neural networks (Boltzmann machines, Basian networks, etc.).

[0078] Random forests are a method for generating multiple decision trees by learning from randomly sampled training data. When using random forests, the system follows the branches of multiple decision trees that have been pre-trained as classifiers, and takes the average (or majority vote) of the results obtained from each decision tree.

[0079] Boosting is a technique for generating a strong classifier by combining multiple weak classifiers. It involves sequentially training simple, weak classifiers to construct a strong classifier.

[0080] SVM is a method for constructing a two-class pattern classifier using linear input elements. SVM learns the parameters of linear input elements using a criterion (hyperplane separation theorem) that finds the margin-maximizing hyperplane that maximizes the distance to each data point from the training data.

[0081] In this embodiment, a multilayer neural network is used as the machine learning algorithm. The neural network includes an input layer for inputting data, an output layer for generating data for the analysis results to be predicted, and one or more hidden layers between the input and output layers. Each layer has multiple nodes (also called units). In detail, in this embodiment, a convolutional neural network (CNN), which is a type of multilayer neural network, is used.

[0082] Other machine learning algorithms may also be used. For example, a Generative Adversarial Network (GAN), which utilizes two competing neural networks, may be adopted as the machine learning algorithm.

[0083] The mathematical model 42 refers to a data structure for predicting the relationship between input data and output data. The mathematical model 42 is constructed by training it using the training dataset 70. The training dataset in this embodiment is a set of input training data 70A and output training data 70B. As shown in Figure 6, the input training data 70A is OCT data of biological tissue captured by the OCT device 10. A large amount of OCT data of biological tissue accumulated in the past is used as the OCT data. In addition to biological tissue from healthy individuals, OCT data of biological tissue with diseases is also used. Furthermore, evaluation values ​​indicating the quality of image data actually generated from each OCT data are used as the output training data 70B. Specifically, as described above, SSI, SNR, or noise level of the biological image are used. By training the mathematical model 42 using this training data, for example, the correlation parameters (e.g., weights) of each input and output are updated. Then, by repeating this training and parameter update, a trained model 40 is constructed.

[0084] Figure 7 shows an example of a trained model 40 constructed by training a mathematical model 42. In this example, the mathematical model 42 is fed a large amount of A-scan data actually acquired by the OCT device 10, and is trained to output image quality information of a one-dimensional image (A-scan image) generated based on the A-scan data. That is, this trained model 40 is trained to take A-scan data as input and output image quality information (e.g., estimated SSI) corresponding to the A-scan data.

[0085] Figure 8 shows an application example of the trained model 40 shown in Figure 7. In this trained model 40, image quality information for one-dimensional images is output for each of the multiple A-scan data that make up one B-scan data. Image quality information for the B-scan data (B-scan image 61) may be calculated using this output of multiple image quality information. In this case, for example, the inference unit 32B may calculate the average value of the multiple image quality information output from the trained model 40 (for example, the average value of the estimated SSI) and output this calculated average value as the image quality information for the B-scan data. Alternatively, the inference unit 32B may output the maximum, minimum, or median value of the multiple image quality information as the image quality information for the B-scan data.

[0086] Figure 9 shows another embodiment of the trained model 40. This trained model 40 is trained to output image quality information about a two-dimensional image (B-scan image 61) generated based on a set of A-scan data that constitutes a single B-scan, when it is input. That is, this trained model 40 takes a set of A-scan data that constitutes the B-scan data as input and outputs image quality information (e.g., estimated SSI) about the B-scan image 61 that is composed of these A-scan data.

[0087] Figure 10 shows another embodiment of the trained model 40. This trained model 40 is trained to output information about the quality of a three-dimensional image 60 when it is input with multiple B-scan data (each B-scan data consists of multiple A-scan data). That is, when this trained model 40 is input with a group of B-scan data that make up a three-dimensional image 60, it outputs image quality information (e.g., estimated SSI) for the three-dimensional image 60. Alternatively, the image quality information for the three-dimensional image 60 may be calculated by inference for each of the multiple B-scan data, and the average value of the calculated multiple image quality information may be output.

[0088] The determination unit 32D is composed of, for example, a CPU. This determination unit 32D determines the quality of the OCT data based on the image quality information output by the inference unit 32B. For example, when calculating the estimated SSI as image quality information, the determination unit 32D determines the quality of the OCT data by comparing the estimated SSI output by the inference unit 32B with a pre-set threshold. This threshold is stored, for example, in the NVM 34. Specifically, if the estimated SSI output by the inference unit 32B is greater than the threshold, the determination unit 32D determines that the OCT data is of high quality. The determination unit 32D then adds this determination result to the image quality information and outputs it to the output unit 32C. On the other hand, if the estimated SSI output by the inference unit 32B is less than the threshold, the determination unit 32D determines that the OCT data is of low quality. The determination unit 32D then adds this determination result to the image quality information and outputs it to the output unit 32C.

[0089] Specifically, when the inference unit 32B outputs image quality information for A scan data (see Figure 7), the determination unit 32D makes a determination on each piece of image quality information. As a result, a quality determination result is added to each piece of A scan data. Also, when the inference unit 32B outputs one piece of image quality information for multiple A scan data (i.e., B scan data) (see Figure 9), the determination unit 32D makes a determination on the image quality information for the B scan data and adds a quality determination result. Furthermore, when the inference unit 32B outputs one piece of image quality information from multiple B scan data (see Figure 10), the determination unit 32D makes a determination on the image quality information for the three-dimensional image 60 composed of these multiple B scan data. The quality determination result for the three-dimensional image 60 is then added to the image quality information. Although this embodiment describes the case in which the determination unit 32D is included, it is not necessarily required to include the determination unit 32D.

[0090] As shown in Figure 1, the control unit 30 is connected to a microphone 36, a monitor 37, and an operation unit 38. For example, the microphone 36 receives voice input from the examiner. The operation unit 38 is operated by the user (examiner) to input various operation instructions to the OCT device 10. The subject can also input evaluations regarding the quality of the biological image (ophthalmic image) displayed on the monitor 37 via the operation unit 38. Various devices such as a mouse, keyboard, touch panel, and foot switch can be used for the operation unit 38. Alternatively, various operation instructions may be input to the OCT device 10 by inputting sound into the microphone 36. In this case, the control unit 30 may determine the type of operation instruction by performing speech recognition processing on the input sound.

[0091] (Regarding Monitor 37) The monitor 37 is an example of a display device that displays various images. In this embodiment, when image data is input from the first processing unit 31, the monitor 37 displays a biological image on the display unit 72 based on the image data. The monitor 37 can also display the analysis results of the image data input from the first processing unit 31 on the display unit 72. Furthermore, when the monitor 37 receives image quality information from the second processing unit 32, it displays information for confirming the image quality (hereinafter referred to as confirmation information) on the display unit 72 based on the image quality information.

[0092] Specifically, in this embodiment, the monitor 37 displays a diagnostic screen 74 that displays a biological image (two-dimensional or three-dimensional tomographic image) and its analysis results, and a confirmation screen 76 that displays confirmation information, on its display unit 72. In the example shown in Figure 11, two diagnostic screens 74 and two confirmation screens 76 are displayed on the display unit 72 of the monitor 37. These diagnostic screens 74 and confirmation screens 76 may be displayed on the display unit 72 of the monitor 37 in a switchable manner. In this case, the switching between the diagnostic screens 74 and confirmation screens 76 may be performed, for example, by an examiner via the operation unit 38, or automatically by the control unit 30.

[0093] In the example shown in Figure 11, a tomographic image of the fundus tissue of the eye being examined is displayed on the diagnostic screen 74 as a biological image. In addition, the analysis results (diagnosis results) obtained by analyzing the image data by the first processing unit 31 are displayed on other diagnostic screens 74.

[0094] Furthermore, one of the confirmation screens 76 of the monitor 37 shown in Figure 11 displays an image visualizing the image quality information output from the second processing unit 32 as confirmation information. More specifically, an image is displayed in which multiple scan lines indicating B-scans are superimposed on a fundus image of the eye under examination taken by the frontal observation optical system. At this time, the monitor 37 displays the B-scan lines 52A, which are determined to be of high quality, and the B-scan lines 52B, which are determined to be of low quality, in different ways. In the example in Figure 11, the high-quality B-scan lines 52A are represented by dashed lines, and the low-quality B-scan lines 52B are represented by solid lines. However, the display method is not limited to solid and dashed lines; it is sufficient that the high-quality B-scan lines 52A and the low-quality B-scan lines 52B are displayed in different ways. For example, the high-quality B-scan lines 52A may be displayed in green and the low-quality B-scan lines 52B in red, or they may be displayed in different colors. Furthermore, while high-quality B scanlines 52A are displayed normally, low-quality B scanlines 52B may be highlighted by blinking or other means. On the other confirmation screen 76, the image quality information values ​​are displayed along with the judgment results from the judgment unit 32D. For example, the estimated SSI of each B scan data is displayed on the other confirmation screen 76 along with the judgment results. That is, the estimated SSI and the judgment results are displayed as confirmation information.

[0095] In this embodiment, the monitor 37 displays a re-shoot button 78, which is operated when the subject's image needs to be retaken. This re-shoot button 78 is operated via the control unit 38, for example, when the examiner determines from the confirmation information displayed on the confirmation screen 76 that a re-shoot of the biological tissue is necessary. Therefore, if the confirmation information displayed on the confirmation screen 76 suggests that the image quality is low, a re-shoot can be performed by operating the re-shoot button 78, even if the actual biological image is still being generated.

[0096] (Regarding the control processing of Medical Information Processing System 1) Next, the processing performed by the medical information processing system 1 according to this embodiment will be described below. Figure 12 is a flowchart showing the overall processing from the medical information processing system 1 to the generation and display of a biological image. The processing in this flowchart is mainly performed by a processor provided in the control unit 30. Furthermore, the following description will explain the case in which ophthalmic tissue (fundus tissue) of the eye under examination is photographed and estimated SSI is output as image quality information.

[0097] When imaging is performed by the OCT device 10 (S10), measurement light is scanned over the ophthalmic tissue, and OCT data is detected by the photodetector 22 (S20). When imaging by the OCT device 10 is completed (S30:YES), the detected OCT data is output to the control unit 30 (S40). Upon receiving the OCT data, the control unit 30 transmits it to the first processing unit 31 and the second processing unit 32, respectively. As a result, as shown in the flowchart of Figure 12, the processing by the first processing unit 31 (S50~S90) and the processing by the second processing unit 32 (S100~S120) are executed in parallel.

[0098] In the first processing unit 31, the calculation unit 31B performs calculations on the received OCT data (S50) to generate image data (S60). The generated image data is then analyzed by the analysis unit 31A (S70). The generated image data, along with the diagnostic results, is output from the first processing unit 31 to the monitor 37 (S80). Upon receiving the image data, the monitor 37 displays the biological image of the ophthalmic tissue (for example, a two-dimensional or three-dimensional tomographic image) and the diagnostic results on the diagnostic screen 74 of the display unit 72 (S90).

[0099] Meanwhile, when OCT data is input to the input unit 32A, the second processing unit 32 executes a quality information calculation process (described later) to calculate image quality information (in this case, estimated SSI) (S100). Once the image quality information is calculated by the quality information calculation process, the image quality information is output to the monitor 37 (S110). The monitor 37, having received the image quality information, then displays confirmation information on the confirmation screen 76 of the display unit 72 (S120).

[0100] As described above, in this embodiment of the medical information processing system 1, when OCT data is acquired by the OCT device 10, the generation of image data based on the OCT data and the inference of image quality based on the OCT data are performed simultaneously. Here, as shown in Figure 13, the time required for the second processing unit 32 (inference unit 32B) to complete the inference of image quality based on the OCT data is shorter than the time required for the first processing unit 31 (calculation unit 31B) to complete the generation of image data based on the OCT data. Therefore, the second processing unit 32 can calculate image quality information before the first processing unit 31 completes the generation of image data. As a result, confirmation information based on image quality information can be displayed on the monitor 37 before displaying the biological image. In other words, the examiner can check the quality of the image to be generated through the monitor 37 without waiting for the image data to be generated. Therefore, the examiner can, for example, make a decision to retake the eye if the image quality is judged to be low at an early stage.

[0101] In the comparative example shown in Figure 13, the image quality is evaluated based on image data generated from OCT data. In this case, the image quality is evaluated after the OCT data has been processed to generate image data. As a result, in the comparative example, it takes time for the results of the image quality evaluation to be displayed on the monitor 37. On the other hand, in this embodiment, image quality information is calculated by inference from the OCT data (RAW data) acquired by the OCT device 10. Therefore, image quality information can be calculated without waiting for the image data generation to be completed, and the confirmation information can be displayed on the monitor 37. As a result, the examiner can grasp the image quality at an earlier stage compared to the comparative example.

[0102] (Regarding the quality information calculation process) Next, the quality information calculation process performed by the second processing unit 32 will be explained with reference to the flowchart in Figure 14. The quality information calculation process shown in Figure 14 is performed by the processor provided in the second processing unit 32. Furthermore, the following explanation will describe the case in which the estimated SSI of the B scan image 61 is calculated from multiple A scan data that constitute the B scan (see Figure 9).

[0103] When the OCT device 10 completes the image acquisition and the OCT data is input to the second processing unit 32 (S200), the inference unit 32B calculates an estimated SSI based on the multiple A scan data that constitute one B scan (S202). That is, as shown in Figure 9, the calculation unit 31B inputs the multiple A scan data that constitute the B scan data into the trained model 40. The trained model 40 is trained to output an estimated SSI for the corresponding B scan image 61 from the multiple A scan data. As a result, the inference unit 32B can obtain one estimated SSI for the corresponding B scan data. Alternatively, as shown in Figure 8, the inference unit 32B may obtain an estimated SSI for each of the multiple A scan data and use the average value as the estimated SSI for the corresponding B scan data.

[0104] Next, the determination unit 32D determines whether the estimated SSI acquired by the inference unit 32B is greater than a predetermined threshold (S204). If the determination unit 32D determines that the estimated SSI is greater than the threshold (S204: YES), it determines that the image quality of the B scan data is high quality (S206) and adds the determination result indicating high image quality to the estimated SSI. On the other hand, if the determination unit 32D determines that the estimated SSI is less than the threshold (S204: NO), it determines that the image quality of the B scan data is low quality (S208) and adds the determination result indicating low quality to the estimated SSI. Then, the output unit 32C outputs the estimated SSI along with the determination result to the monitor 37 (S210).

[0105] Next, the inference unit 32B determines whether or not an estimated SSI has been obtained for all B scans (S212). If an estimated SSI has been obtained for all B scans (S212: YES), the image quality information calculation process ends. On the other hand, if an estimated SSI has not been obtained for all B scans (S212: NO), the process returns to step S202, and the calculation of the estimated SSI for the next B scan is repeated (S202).

[0106] As described above, the image quality information calculation process shown in this embodiment acquires image quality information for each B scan. Therefore, confirmation information for each B scan can be displayed on the monitor 37. Moreover, for each B scan, information representing the quality judgment result is added to the image quality information. Therefore, for example, as shown in Figure 11, it is possible to display a low-quality B scan line 52B on the monitor 37 in a different manner than a high-quality B scan line 52A. This allows the examiner to visually grasp the quality of the image immediately. Furthermore, by displaying the image quality information (estimated SSI) of each B scan along with its judgment result on the monitor 37, the examiner can accurately identify low-quality B scans.

[0107] Based on the above, the medical information processing system 1 according to this embodiment can estimate the quality of the generated image before the generation of the biological image is completed, and inform the examiner of the estimation result. For example, if the eye under examination moves during imaging by the OCT device 10, the examiner can understand that the image based on the acquired OCT data will be of low quality before the image is generated. As a result, the examiner can decide to re-imaging the eye under examination without waiting for the diagnostic image to be generated.

[0108] (Example of transformation) It should be noted that the medical information processing system 1 and processing device relating to this disclosure are not limited to the embodiments described above, and various modifications are possible. For example, in the embodiments described above, the calculation of image quality information by the second processing unit 32 is performed after the OCT device 10 has finished capturing images of the subject's biological tissue (see Figure 13). However, as shown in Figure 15, the inference by the second processing unit 32 may be started before the OCT device 10 has finished capturing images. In this case, image quality information can be acquired earlier than in the embodiments, and confirmation information can be displayed on the monitor 37. In the case of the modified example, the inference by the second processing unit 32 may be started, for example, at the timing when the OCT device 10 has completed the first B scan and the B scan data has been input to the input unit 32A.

[0109] Furthermore, in this embodiment, an example was given in which image quality information for each B scan is acquired and a confirmation image of the B scan is displayed on the monitor 37. However, for example, image quality information for each A scan may be acquired, and confirmation information for each A scan may be displayed based on the acquired image quality information. In this case, since the image quality information for each A scan would be enormous, for example, an image graphed in chronological order of each A scan and its quality confirmation information (e.g., estimated SSI) may be displayed on the monitor 37 as confirmation information. Alternatively, a frontal image (e.g., a so-called "Enface image") in which multiple A scans are arranged in two dimensions may be displayed on the monitor 37, and each A scan may be displayed in a different color according to its quality. For example, high-quality A scans may be displayed in green, and low-quality A scans may be displayed in red. As a result, the two-dimensional distribution of the quality of each A scan can be easily grasped.

[0110] In the embodiment described above, an example was given in which the examiner judges to perform a rescan of the eye under examination based on the confirmation image displayed on the monitor 37. However, the OCT device 10 may be configured to perform the rescan automatically. Figure 16 is a flowchart showing the automatic rescanning process performed by the second processing unit 32 (processing unit) in relation to the example of transformation. In this flowchart, an example is given in which image quality information (estimated SSI) is calculated for each B scan, similar to the embodiment described above. Furthermore, the same reference numerals are used for processes identical to those in the embodiment described above, and their explanation is omitted.

[0111] In the automatic rescanning process, when OCT data is input to the second processing unit 32 (S200), the inference unit 32B calculates image quality information based on each B scan data (multiple A scan data) as in the embodiment (S202). Next, the determination unit 32D determines whether the image quality information calculated by the inference unit 32B is greater than a threshold (S204). If the determination unit 32D determines that the quality of the B scan data is low (S204: NO), the B scan determined to be of low quality is stored in a predetermined memory (S300). This memory may be the memory provided in the second processing unit 32, or it may be the NVM 34 mentioned above. Then, the calculation of image quality information and the determination of quality are performed for all B scans (S302).

[0112] Once the quality of all B scans has been determined (S302: YES), the output unit 32C refers to the memory and sends a command to the OCT device 10 to re-acquire the B scans that were determined to be of low quality (S304). As a result, the OCT device 10 automatically re-acquires the B scans that were determined to be of low quality. In this way, the automatic re-acquisition process automatically re-acquisitions the B scans that were determined to be of low quality without the examiner having to make a judgment. Therefore, only high-quality biological images can be automatically obtained without burdening the examiner. When re-acquisition, the alignment (to avoid cataract opacity), focus, and OPL (Optical Path Length) of the OCT device 10 may be optimized to increase the estimated SSI. Furthermore, the automatic re-acquisition process described above can also be applied when motion contrast images (for example, so-called Angio images) are acquired by performing multiple B scans on the same scan line.

[0113] In the embodiments and modifications described above, the quality of the OCT data was displayed on the monitor 37 by displaying indicator values ​​such as estimated SSI on the monitor 37, or by displaying low-quality B scanlines in a different manner from high-quality B scanlines. However, the image quality information is not limited to these, and other information may be used as long as it relates to the quality of the image generated based on the OCT data.

[0114] For example, the image data generated from the OCT data may be calculated by inference. In this case, when OCT data is input, the trained model 40 is constructed to output image data corresponding to the OCT data. More specifically, for example, when B-scan data (multiple A-scan data) is input to the trained model 40, an estimated value of the B-scan image 61 data generated from the B-scan data (hereinafter referred to as the estimated image) is output. In this way, by enabling the acquisition of the B-scan image 61 from the B-scan data by inference, the image quality can be checked based on the B-scan image 61 obtained by inference, without actually generating the B-scan image 61.

[0115] Furthermore, as shown in Figure 17, by obtaining estimated images from OCT data, these estimated images can be used as complements to biological images, as described below. For example, estimated images may be obtained from a portion of multiple B-scan data, while actual B-scan images 61 (hereinafter referred to as generated images) may be generated from the remaining B-scan data. By combining the estimated images and generated images and displaying them on the monitor 37, the examiner can check the image quality at an earlier stage compared to generating the entire three-dimensional image 60. In the example in Figure 17, the three-dimensional image 60 is formed by combining estimated images and generated images so that they are arranged alternately in the Y direction.

[0116] In the embodiment described above, image quality information is calculated by inputting OCT data into a pre-trained model 40. However, the pre-trained model 40 may be continuously trained (updated), for example, based on feedback from the examiner. For example, the examiner may input an evaluation of the image quality of the biological image displayed on the monitor 37 via the operation unit 38 (e.g., high quality, low quality, medium quality, etc.). By feeding the input evaluation back to the pre-trained model 40, it becomes possible to train the pre-trained model 40 in real time. As a result, quality evaluations tailored to the examiner's preferences are reflected, making it possible to customize the pre-trained model 40 for each examiner.

[0117] In the embodiment described above, the first processing unit generates a biological image, and the second processing unit infers image quality information. However, for example, the analysis unit 31A (CPU) of the first processing unit may perform image quality information inference along with biological image generation. That is, a single processing unit may perform both biological image generation and image quality information inference. In this case, a single processing unit obtains image quality information by inference based on OCT data. Then, if the OCT data is of high quality based on the image quality information, the processing unit may generate a biological image by calculation based on the OCT data.

[0118] Figure 18 is a flowchart illustrating the process when image quality inference and biological image generation are performed by a single processing unit. The flowchart in Figure 18 illustrates a case where image quality inference and B-scan image generation are performed sequentially based on multiple B-scan data. Furthermore, the case where estimated SSI is output as image quality information is explained.

[0119] When imaging begins, the OCT device 10 performs the first B scan (S400). Once B scan data is acquired from the first B scan, the B scan data is input to the processing unit (S402). The processing unit (CPU) calculates the estimated SSI from the input B scan data by inference (S404). If the estimated SSI is higher than the threshold (S406: YES), the first B scan data is determined to be of high quality (S408), and the processing unit generates a B scan image based on the B scan data (S410).

[0120] On the other hand, if the calculated estimated SSI is smaller than the threshold (S406: NO), the first B-scan data is determined to be of low quality (S412). If it is determined to be of low quality, the first scan data is discarded, and the B-scan is performed again (S400). In this way, if the acquired B-scan data is determined to be of low quality, the B-scan data is recaptured without generating an image based on that B-scan data. In other words, a B-scan image is generated only if the acquired B-scan data is of high quality. Therefore, even with a single processing unit, it is possible to efficiently perform both image quality information inference and biological image generation.

[0121] In S414, it is determined whether all B-scans have been performed. If all B-scans have not been performed (S414: NO), the second and subsequent B-scans are performed (S400). On the other hand, if all B-scans have been performed (S414: YES), this process ends. Here, the determination process in S414 is performed without waiting for the image generation in S410 to be completed. That is, for B-scan data determined to be of high quality, the processing unit performs image data calculations, and the OCT device 10 acquires (captures) the next B-scan data in parallel (S400). Therefore, processes such as B-scan data acquisition, quality evaluation, and image generation proceed smoothly without delay. [Explanation of Symbols]

[0122] 1. Medical Information Processing System 10 OCT device 11 OCT light source 12. Coupler (branched optical element) 14 Scanning Unit 16 Irradiation optical system 22 Photodetector 30 Control Unit (Processing Unit) 31. First Processing Unit 32 Second Processing Unit 32A Input Section 32B Reasoning part 32C output section 37. Monitor (display device) 40 pre-trained models 70A Input training data 70B Output Training Data

Claims

1. A medical information processing system that acquires biological images, which are images of the subject's tissue, OCT device and Processing device and A display device, The OCT device is, OCT light source and A branching optical element that splits the light emitted from the OCT light source into measurement light and reference light, An irradiation optical system that irradiates the measurement light, which has been branched by the branching optical element, onto the tissue of the subject, The system comprises a light-receiving element that detects the interference signal of the measurement light reflected by the aforementioned structure and the reference light branched by the branched optical element as OCT data, The aforementioned processing apparatus is A first processing unit generates a biological image of the tissue based on the OCT data detected by the light-receiving element, The system includes a second processing unit that outputs image quality information obtained by inferring the quality of the biological image based on the OCT data, before the first processing unit completes the generation of a biological image of the tissue based on the OCT data, The display device displays confirmation information for the examiner to confirm the quality of the biological image of the tissue, based on the image quality information output from the second processing unit. The medical information processing system is characterized in that the second processing unit outputs image quality information using a pre-trained model that has been trained by machine learning to output the image quality information when the OCT data of the subject's tissue is input.

2. The light-receiving element detects the A-scan data as the OCT data for generating an A-scan image, which is a one-dimensional biological image of the tissue. The medical information processing system according to claim 1, wherein the trained model is pre-trained to output image quality information relating to the quality of the A-scan image generated based on the A-scan data when the A-scan data is input.

3. The medical information processing system according to claim 2, wherein the display device displays the confirmation information for a plurality of A scan data based on the plurality of image quality information output from the second processing unit.

4. The light-receiving element detects a plurality of A-scan data that constitute B-scan data for generating a B-scan image, which is a two-dimensional biological image of the tissue, as the OCT data. The second processing unit calculates the image quality information corresponding to the B-scan data based on the plurality of image quality information corresponding to the plurality of A-scan data constituting the B-scan data, which is output from the trained model. The medical information processing system according to claim 2, wherein the display device displays the confirmation information for the B-scan data based on the image quality information corresponding to the B-scan data output from the second processing unit.

5. The light-receiving element detects a plurality of A-scan data that constitute B-scan data for generating a B-scan image, which is a two-dimensional biological image of the tissue, as the OCT data. The medical information processing system according to claim 1, wherein the trained model is pre-trained to output image quality information relating to the quality of the B scan image generated based on the plurality of A scan data when the plurality of A scan data is input.

6. The medical information processing system according to any one of claims 1 to 5, wherein the trained model is pre-trained to output an estimated value of the signal intensity of a biological image generated based on the OCT data as image quality information.

7. The medical information processing system according to claim 6, wherein the display device displays the confirmation information of the OCT data corresponding to the estimated value smaller than a preset threshold in a different manner from the confirmation information of the OCT data corresponding to the estimated value larger than the threshold.

8. A processing device for processing OCT data obtained from an OCT device that images the tissue of a subject, An input unit for acquiring OCT data from the OCT device, Before the generation of a biological image of the tissue from the OCT data acquired by the input unit is completed, an inference unit outputs image quality information obtained by estimating the quality of the biological image based on the OCT data, The system includes an output unit that outputs the image quality information output by the inference unit to a display device, The inference unit is a processing unit that outputs image quality information using a pre-trained model that has been trained by machine learning to output the image quality information when the OCT data of the subject's tissue is input.

9. The input unit acquires A-scan data from the OCT device as OCT data for generating an A-scan image, which is a one-dimensional biological image of the tissue. The processing apparatus according to claim 8, wherein the trained model is pre-trained to output image quality information relating to the quality of the A-scan image generated based on the A-scan data when the A-scan data is input.

10. The input unit acquires a plurality of A-scan data from the OCT device as OCT data, which constitutes B-scan data for generating a B-scan image, which is a two-dimensional biological image of the tissue. The processing apparatus according to claim 9, wherein the inference unit calculates the image quality information corresponding to the B-scan data based on the plurality of image quality information corresponding to the plurality of A-scan data constituting the B-scan data, which is output from the trained model.

11. The input unit acquires a plurality of A-scan data from the OCT device as OCT data, which constitutes B-scan data for generating a B-scan image, which is a two-dimensional biological image of the tissue. The processing apparatus according to claim 8, wherein the trained model is pre-trained to output image quality information relating to the quality of the B scan image generated based on the plurality of A scan data when the plurality of A scan data is input.

12. The processing apparatus according to claim 8, wherein the trained model is pre-trained to output an estimated value of the signal intensity of a biological image generated based on the OCT data as the image quality information.

13. The processing apparatus according to claim 12, wherein if the estimated value of the OCT data output by the inference unit is lower than a preset threshold, the output unit outputs a signal instructing the OCT device to acquire the OCT data again.

14. The processing apparatus according to any one of claims 8 to 13, wherein the trained model is trained using a plurality of OCT data relating to the tissues of a plurality of subjects acquired in advance as input training data, and the signal intensities of a plurality of biological images generated based on the plurality of OCT data as output training data.

15. A processing device for processing OCT data obtained from an OCT device that images the tissue of a subject, An input unit that acquires multiple B-scan data as OCT data for generating multiple B-scan images, which are two-dimensional biological images of the tissue, from the OCT device, Before the generation of a three-dimensional biological image of the tissue from the plurality of B-scan data acquired by the input unit is completed, an inference unit outputs image quality information obtained by inputting input data related to at least some of the B-scan data from the plurality of B-scan data, thereby estimating the quality of the three-dimensional biological image. The system includes an output unit that outputs the image quality information output by the inference unit to a display device, The inference unit is a processing unit that outputs image quality information using a pre-trained model that has been trained by machine learning to output the image quality information when the input data is input.

16. The processing apparatus according to claim 15, wherein the input data is a B-scan image generated from at least a portion of the B-scan data among the plurality of B-scan data.

Images