Medical information processing program and medical information processing apparatus

Abstract

This medical information processing device includes a control unit that acquires target data including data of a medical image relating to a person to be diagnosed, and information about the medical image imaging timing (S1). The control unit extracts, as similar case data, a case data item that includes a medical image similar to the medical image included in the target data, from a plurality of case data items (S3). The control unit matches the time axis of the medical image imaging timing between a plurality of data items (S7, S10, S14). The control unit outputs medical information based on the similar case data, with the time axes being matched (S7, S11, S15, S18, S22).

Description

Technical Field

[0001] The present disclosure relates to a medical information processing program and a medical information processing device for outputting medical information useful for diagnosis based on case data similar to the data of a person to be diagnosed among a plurality of case data.

[0002] Conventionally, various techniques for presenting medical information useful for diagnosis to a user have been proposed. For example, the ophthalmic information processing device disclosed in Patent Document 1 facilitates the follow-up observation of an eye to be examined by aligning layer thickness information based on a plurality of OCT data obtained on different days.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] It is also conceivable to extract case data similar to the data of a person to be diagnosed from a plurality of case data including a plurality of medical images taken at different timings, and output medical information based on the extracted case data. In this case, since the transition of the state of the patient's disease is grasped from the plurality of medical images included in the extracted case data, the transition of the disease of the person to be diagnosed is appropriately predicted by the output medical information.

[0005] However, both the plurality of medical images included in the case data and the medical images of the person to be diagnosed are taken at various timings. Therefore, if medical information is not output after considering the timing at which each medical image was taken, there is a possibility that the diagnosis by a doctor may not be appropriately assisted.

[0006] An object of the present disclosure is to provide a medical information processing program and a medical information processing device capable of presenting more useful medical information to a user.

[0007] A medical information processing program provided by a typical embodiment in the present disclosure is a medical information processing program executed in a medical information processing device that outputs medical information useful for diagnosis based on case data similar to the data of a person to be diagnosed among a plurality of case data. Each of the plurality of case data includes data of a plurality of medical images taken at different timings and information on the imaging timing of each medical image. , treatment information regarding the treatment performed on the patient, and They are stored in a database in a state including them. the treatment information included in the case data includes information indicating the treatment timing when the treatment was executed or started. When the medical information processing program is executed by the control unit of the medical information processing device, a target data acquisition step of acquiring target data including data of at least one medical image regarding the person to be diagnosed and information on the imaging timing of the medical image, a similar case data extraction step of extracting, as similar case data, the case data including a medical image similar to at least one medical image included in the target data from the plurality of case data, a timeline matching step of matching the timelines of the imaging timings of the medical images between the similar case data and the target data, type and a medical information output step of outputting medical information based on the similar case data in a state where the timelines are matched are executed by the medical information processing device. causing the medical image included in the target data and the plurality of medical images included in the similar case data to be displayed on a display unit, and the imaging timing of the medical image of the target data, the imaging timing and treatment timing of the plurality of medical images of the similar case data, In a state where the timelines are matched by displaying them on the time axis, A medical information processing device provided by a typical embodiment in the present disclosure is a medical information processing device that outputs medical information useful for diagnosis based on case data similar to the data of a person to be diagnosed among a plurality of case data. Each of the plurality of case data includes data of a plurality of medical images taken at different timings and information on the imaging timing of each medical image.

[0008] They are stored in a database in a state including them. , treatment information regarding the treatment performed on the patient, and They are stored in a database in a state including them. the treatment information included in the case data includes information indicating the treatment timing when the treatment was executed or started. The control unit of the medical information processing device performs a target data acquisition step of acquiring target data including data of at least one medical image regarding a subject to be diagnosed and information on the imaging timing of the medical image, a similar case data extraction step of extracting, as similar case data, the case data including a medical image similar to at least one medical image included in the target data from the plurality of case data, a time axis matching step of matching the time axes of the imaging timings of the medical images among the similar case data, the target data, and at least one of the plurality of similar case data, and a medical information output step of outputting medical information based on the similar case data. causing the medical image included in the target data and the plurality of medical images included in the similar case data to be displayed on a display unit, and the imaging timing of the medical image of the target data, the imaging timing and treatment timing of the plurality of medical images of the similar case data, In a state where the time axes are matched by displaying them on the time axis, it performs a medical information output step of outputting medical information based on the similar case data.

[0009] According to the medical information processing program and the medical information processing method in the present disclosure, more useful medical information is presented to the user.

[0010] The medical information processing program exemplified in the present disclosure is executed in a medical information processing apparatus. The medical information processing apparatus outputs medical information useful for diagnosis based on case data similar to the data of the person to be diagnosed among a plurality of case data. Each of the plurality of case data is stored in a database in a state including data of a plurality of medical images taken at different timings and information on the imaging timings of the respective medical images. The control unit of the medical information processing apparatus executes a target data acquisition step, a similar case data extraction step, a timeline matching step, and a medical information output step. In the target data acquisition step, the control unit acquires target data including data of at least one medical image regarding the person to be diagnosed and information on the imaging timing of the medical image. In the similar case data extraction step, the control unit extracts case data (similar case data) including a medical image similar to at least one medical image included in the target data from the plurality of case data. In the timeline matching step, the control unit matches the timelines of the imaging timings of the medical images between the similar case data and the target data and between at least one of the plurality of similar case data. In the medical information output step, the control unit outputs medical information based on the extracted case data with the timelines matched.

[0011] According to the medical information processing apparatus exemplified in the present disclosure, medical information is output in a state where the timelines of the imaging timings of medical images are matched between a plurality of data (at least one of between similar case data and target data and between a plurality of similar case data). Therefore, a user (such as a doctor etc.) who checks the output medical information can appropriately predict the transition of the disease of the person to be diagnosed after grasping the timing at which each medical image was taken. Thus, more useful medical information is presented to the user.

[0012] Note that the method for extracting similar case data can be set as appropriate. For example, the control unit may extract values indicating the feature amounts of each medical image (e.g., SIFT (Scale Invariant Feature Transform), etc.), and extract case data including medical images with small differences in feature amounts from the medical images of the target data as similar case data. Further, the control unit may input a medical image into a mathematical model trained by a machine learning algorithm to obtain the similarity degree of the medical image, and extract similar case data based on the obtained result. Further, when the image quality of the medical image included in the case data is less than the threshold value, the control unit may exclude it from the extraction target, or may warn the user that the image quality is low.

[0013] In the time axis matching step, the control unit may match the time axes of the shooting timings of the medical images among a plurality of data according to an instruction input from the user. In this case, the user can output appropriate medical information to the medical information processing device in a state where the time axes among the plurality of data are matched to a desired state.

[0014] Note that a specific method for matching the time axis according to an instruction from the user can be selected as appropriate. For example, the time axes indicating the shooting timings of the medical images of each of the plurality of data may be displayed on the display unit. The user may input an instruction to adjust the time axis while checking the time axis indicating the shooting timing to the medical information processing device. The control unit may match the time axis according to the input adjustment instruction. In this case, the control unit may also display the medical image on the display unit together with the time axis. Further, the user may input an instruction to specify a criterion for matching the time axis. For example, the control unit may input an instruction to specify, as a reference image, a medical image whose shooting timing is to be matched on the time axis among the medical images of each of the plurality of data. In this case, the control unit may match the time axes so that the shooting timings of the medical images specified as the reference images match each other.

[0015] In the time axis alignment step, the control unit may align the time axes by aligning the imaging timings of the medical images with the highest similarity among the medical images included in each of the plurality of data. The disease states in the two medical images with the highest similarity are likely to be approximated. Therefore, by aligning the time axes so that the imaging timings of the two medical images with the highest similarity match, the user can more appropriately grasp the transition of the disease based on the medical information.

[0016] The case data may include information (treatment information) regarding the treatment (for example, medication, surgery, or procedure, etc.) performed on the patient. In this case, the medical information processing apparatus can output more appropriate medical information based on the treatment information included in the case data.

[0017] The specific content of the treatment information can be appropriately selected. For example, the treatment data may include at least any one of information indicating whether a treatment has been performed on the patient, information indicating the content of the treatment performed on the patient, and information indicating the treatment timing when the treatment has been performed or started on the patient.

[0018] In the target data acquisition step, the control unit extracts case data (hereinafter referred to as "similar case data") including a medical image similar to at least one medical image included in the target data from among one or a plurality of case data groups classified within the plurality of case data according to at least either the presence or absence of the treatment indicated by the treatment information and the content of the treatment performed. In this case, after appropriately extracting the case data according to at least either the presence or absence of the treatment and the content of the treatment, the medical information is output. Therefore, the user can appropriately predict the transition of the disease of the diagnostic subject according to the content of the treatment and the like.

[0019] Note that the control unit may extract similar case data from one of the multiple case data groups. Further, the control unit may extract at least one piece of similar case data from each of the multiple case data. For example, by extracting similar case data from each of the case data group of patients who have received treatment and the case data group of patients who have not received treatment, the progression of the disease according to the presence or absence of treatment can be appropriately predicted. Also, by extracting similar case data from each of the multiple treatment data groups with different treatment contents, the progression of the disease according to the treatment content can be appropriately predicted.

[0020] The treatment information included in the case data may include information indicating the treatment timing when the treatment was executed or started. In the time axis matching step, the control unit may match the time axes by matching the treatment timings between the multiple data. In this case, based on the treatment timing, the progression of the disease of the diagnostic subject due to treatment can be appropriately predicted.

[0021] Note that the treatment timing for the diagnostic subject may be the actual treatment timing for the diagnostic subject, or may be the treatment timing when it is assumed that future treatment will be executed for the diagnostic subject.

[0022] Note that it is also possible to change the specific method for matching the time axes of the multiple data. For example, the control unit may specify, for each of the multiple data, the timing (change timing) when the change in the feature amount of the multiple medical images becomes equal to or greater than the threshold. The control unit may match the time axes by matching the change timings of the multiple data. In this case, based on the change timing, the progression of the disease can be appropriately predicted. For example, the control unit may specify the change timing when the change in the degree of progression of the disease (degree of progression of deterioration) based on the feature amounts of the multiple medical images, etc., becomes equal to or greater than the threshold, and match the change timings of the multiple data. In this case, the user can easily compare the multiple data based on the timing when the degree of progression of the disease has rapidly deteriorated.

[0023] Also, when extracting similar case data from a plurality of case data, the control unit may extract similar case data by comparing the similarity between the medical images included in the case data that are before or after the treatment / change timing and the medical images of the target data. In this case, similar case data is more appropriately extracted considering the treatment / change timing.

[0024] In the similar case data extraction step, when the target data includes a plurality of medical images, the control unit may extract, as similar case data, case data including a plurality of medical images taken at an imaging interval whose difference from the imaging interval of the plurality of medical images in the target data is equal to or less than a threshold value, where the plurality of medical images are similar to each other. In this case, the plurality of medical images of the extracted similar case data include medical images taken at an interval close to the imaging interval of the plurality of medical images of the target data. Therefore, the user can more appropriately predict the progression of the disease based on case data in which the imaging intervals of the plurality of medical images are close and the time axes also match. Note that the threshold value for the difference in the imaging intervals between the medical images of the target data and the medical images of the case data may be set in advance or may be set according to an instruction input by the user.

[0025] Note that when the target data includes a plurality of medical images, the control unit can extract similar case data in various ways. For example, the control unit may set one or more medical images to be judged for similarity with the case data among the plurality of medical images included in the target data according to an instruction input by the user. The control unit may extract similar case data by comparing the similarity between the medical images to be judged and the medical images of the case data.

[0026] In the medical information output step, the control unit may cause the display unit to display, as medical information, both the medical images included in each of the plurality of data and the time axis indicating the imaging timing of the medical images, with the time axes aligned. In this case, the user can appropriately grasp the imaging timing of the plurality of medical images displayed on the display unit with the time axes of the plurality of data aligned. Therefore, the user can more easily predict the progression of the disease.

[0027] Note that the control unit may cause the display unit to display the medical images included in each of the plurality of similar case data together with the time axis. Further, the control unit may cause the display unit to display the medical images included in the similar case data and the medical images included in the target data together with the time axis. In this case, the user can appropriately compare the medical images of the target data and the medical images of the similar case data after grasping the imaging timing according to the time axis.

[0028] The control unit may generate a predicted image of the subject to be diagnosed based on the medical images of the target data and the medical images of the similar case data that were taken at a timing later than the same timing as the imaging timing of the medical images of the target data on the aligned time axis among the medical images included in the similar case data. In the medical information output step, the control unit may cause the display unit to display the predicted image generated with the time axes aligned, as medical information. In this case, an appropriate predicted image is generated and displayed in consideration of the imaging timing based on the medical images of each of the similar case data and the target data. Therefore, the user can more appropriately predict the progression of the disease.

[0029] The control unit may further execute a progress acquisition step and a graph generation step. In the progress acquisition step, the control unit acquires information on the progress of the disease depicted in each of the plurality of medical images. In the graph generation step, the control unit generates a progress transition graph showing the progress transition in the similar case data and the target data, or in each of the plurality of similar case data, with the time axes between the plurality of data being aligned. In the medical information output step, the control unit may cause the display unit to display the progress transition graph generated with the time axes aligned as medical information. In this case, the user can easily grasp the progress transition of the disease in the plurality of data. Further, since the progress transition graph is generated with the time axes between the plurality of data being aligned, the user can appropriately compare the progress transition graphs of the plurality of data.

[0030] At least some of the plurality of case data may include heterogeneous data acquired by a device of a different type from the medical image capturing device that captured the medical images included in the target data regarding the subject to be diagnosed. In the medical information output step, when the extracted similar case data includes heterogeneous data, the control unit may output the heterogeneous data as medical information. The heterogeneous data may be, for example, an image captured by a capturing device of a different type from the medical image capturing device that captured the medical images included in the target data (i.e., the medical images referred to when extracting the similar case data). Further, the heterogeneous data may be an examination result acquired by an examination device that examines the patient (for example, in the ophthalmology field, at least one of the examination results such as the visual acuity, axial length, intraocular pressure, and visual field of the eye to be examined). Further, the heterogeneous data may be displayed together with the medical images included in the similar case data, or the heterogeneous data may be output (for example, displayed) alone.

[0031] In this case, since the user can perform a diagnosis or the like after checking the heterogeneous data included in the similar case data, the transition of the disease of the subject to be diagnosed can be predicted more appropriately. Further, even when the heterogeneous data regarding the subject to be diagnosed has not yet been acquired, the user can perform a diagnosis or the like of the subject to be diagnosed more appropriately by checking the heterogeneous data included in the similar case data.

Brief Description of the Drawings

[0032]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0033] Hereinafter, one of the typical embodiments in the present disclosure will be described with reference to the drawings. In this embodiment, a medical information processing system 1 including a server 10, a plurality of medical information processing devices 20 used at each site, and a medical image capturing device 30 that supplies data including medical images to the medical information processing device 20 is exemplified. However, the configurations of the medical information processing system and the medical image processing device are not limited to the configurations exemplified in this embodiment. For example, in this embodiment, the storage device of the server 10 is used as a database for storing a plurality of case data. However, when the case data is stored in the storage device 23 of the medical information processing device 20, it is also possible to omit the server 10. Further, the server 10 or the medical image capturing device 30 may function as a medical information processing device. Further, two or more devices (for example, a server, a terminal device (personal computer or mobile terminal, etc.), and a medical image capturing device, etc.) may cooperate to function as a medical information processing device.

[0034] Referring to FIG. 1, the system configuration of the medical information processing system 1 in this embodiment will be described. As described above, the medical information processing system 1 of this embodiment includes a server 10 and a plurality of medical information processing devices 20 used at each site (for example, a hospital, a health examination facility, etc.). In FIG. 1, a medical information processing device 20A used at site A and a medical information processing device 20B used at site B are exemplified.

[0035] Server 10 provides various data and the like to the connected devices (medical information processing device 20 in this embodiment). In this embodiment, a server of a manufacturer that provides cloud services (so-called cloud server) is used as server 10. However, it goes without saying that a server other than the cloud server may be used. Server 10 includes a control unit 11 that performs various process controls and a communication I / F 14. The control unit 11 includes a CPU 12 that is a controller for controlling and a storage device 13 that can store programs, data, and the like. In this embodiment, the storage device 13 of server 10 is used as a database for storing case data described later. The communication I / F 14 connects server 10 to an external device (for example, medical information processing device 20) via a network 5 (for example, the Internet).

[0036] The medical information processing device 20 is used by users (for example, doctors and medical technicians, etc.) at each site. The medical information processing device 20 in this embodiment is a personal computer, but a mobile terminal such as a smartphone or a tablet terminal may be used as the medical information processing device. The medical information processing device 20 includes a control unit 21 that performs various control processes and a communication I / F 24. The control unit 21 includes a CPU 22 that is a controller for controlling and a storage device 23 that can store programs, data, and the like. The storage device 23 stores a medical information processing program for executing various processes described later. Also, the communication I / F 24 connects the medical information processing device 20 to an external device (for example, server 10) via the network 5. For example, the medical information processing device 20 receives (acquires) case data from server 10. Also, the medical information processing device 20 transmits (outputs) case data to server 10.

[0037] The medical information processing device 20 is connected to an operation unit 25 and a display unit 26. The operation unit 25 is operated by a user to input various instructions to the medical information processing device 20. At least one of, for example, a keyboard, a mouse, a touch panel, etc. can be used for the operation unit 25. Note that, together with or instead of the operation unit 25, a microphone or the like for inputting various instructions may be used. The display unit 26 is a device (for example, a monitor or a projector, etc.) capable of displaying various images.

[0038] At least one of the plurality of medical information processing devices 20 can exchange data (for example, data of medical images, etc.) with one or more medical image capturing devices 30 that capture medical images of a patient. The method by which the medical information processing device 20 exchanges data, etc. with the medical image capturing device 30 can be appropriately selected. For example, the medical information processing device 20 may exchange data and control signals with the medical image capturing device 30 by at least one of wired communication, wireless communication, a removable storage medium (for example, a USB memory), etc.

[0039] Various devices can be used for the medical image capturing device 30. As an example, the medical image capturing device 30 used in the present embodiment includes an OCT device capable of acquiring tomographic images and frontal images of the tissue of the eye to be examined (the fundus in the present embodiment). However, an ophthalmic imaging device other than the OCT device (for example, at least one of a fundus camera, a scanning laser ophthalmoscope (SLO), a corneal shape measuring device, etc.) may be used. Further, a medical image capturing device that captures the tissue of a patient other than the eye to be examined may be used.

[0040] The medical imaging device 30 includes a control unit 31 that performs various control processes and an imaging unit 35. The control unit 31 includes a CPU 32 that is a controller for controlling, and a storage device 33 that can store programs, data, and the like. The imaging unit 35 includes various components necessary for the medical imaging device 30 to capture a medical image of a patient. For example, when an OCT device is used as the medical imaging device 30, the imaging unit 35 includes an OCT light source, a scanning unit for scanning the OCT light, an optical system for irradiating the OCT light to the eye to be examined, a light receiving element for receiving the light reflected by the tissue of the eye to be examined, and the like.

[0041] Referring to FIGS. 2 to 6, an example of medical information processing in the present embodiment will be described. The medical information processing illustrated in FIG. 2 is executed by the CPU 22 of the medical information processing device 20 according to a medical information processing program stored in the storage device 23 of the medical information processing device 20. However, as described above, the medical information processing may be executed by a control unit of another device (for example, the CPU 12 of the server 10 or the CPU 32 of the medical imaging device 30, etc.). Further, control units of a plurality of devices (for example, the CPU 22 of the medical information processing device 20 and the CPU 12 of the server 10, etc.) may cooperate to execute the medical information processing.

[0042] First, the CPU 22 acquires target data 40 regarding the person to be diagnosed (S1). As shown in FIGS. 3 and 4, the target data 40 in this embodiment includes at least one medical image 41 taken by the medical imaging device 30 of the person to be diagnosed and information on the imaging timing (in this embodiment, information on the imaging date and time) when each medical image 41 was taken. The CPU 22 may acquire the target data 40 from the medical imaging device 30 via communication or a removable storage medium or the like. Further, the CPU 22 may acquire, as the target data 40, both the past target data of the person to be diagnosed already stored in the storage device 23 and the medical image 41 newly taken by the medical imaging device 30 and the information on the imaging timing. Although details will be described later, the medical information processing device 20 extracts, as similar case data, case data similar to the target data 40 from a plurality of case data stored in the database.

[0043] Here, with reference to FIGS. 3 and 4, a plurality of case data 50 stored in the database will be described. In this embodiment, the storage device 13 of the server 10 is used as the database for storing the plurality of case data 50. However, the database for storing the case data 50 may be another storage device (for example, the storage device 23 of the medical information processing device 20 or the like). The case data 50 includes a plurality of medical images 51 taken at different timings by the medical imaging device 30 and information on the imaging timing of each medical image 51. The plurality of case data 50 are stored in the database.

[0044] Furthermore, as shown in FIG. 4, the case data 50 in this embodiment includes information regarding the treatment performed on the patient (hereinafter referred to as "treatment information"). In this embodiment, the treatment information indicates the presence or absence of treatment for the patient, the timing when the treatment was executed or started (hereinafter referred to as "treatment timing"), and the content of the treatment performed (for example, at least any one of the type of surgery performed, the type of drug administered, the method of drug administration, the type of procedure performed, etc.).

[0045] Each of the plurality of case data 50 stored in the database is classified into one of a plurality of case data groups according to the presence or absence of treatment indicated by the treatment information and the content of the treatment performed. That is, in the present embodiment, each of the plurality of case data 50 is classified into either a case data group of a patient who has not received treatment or a case data group of a patient who has received treatment. Further, the case data group of the patient who has received treatment is further classified in detail according to the content of the treatment performed. Although details will be described later, the medical information processing apparatus 20 can also extract case data 50 similar to the target data 40 according to the classification of the case data group.

[0046] In addition, at least some of the plurality of case data stored in the database may include data related to the patient (hereinafter referred to as "heterogeneous data") acquired by an apparatus of a type different from the medical image capturing apparatus 30 that captures the medical image 41 of the target data 40. The heterogeneous data may include data of an image captured by an imaging device of a type different from the above-described medical image capturing device 30 (in this embodiment, an OCT device) (for example, at least one of a fundus camera, a scanning laser ophthalmoscope (SLO), a corneal shape measuring device, etc.). Further, the heterogeneous data may include data of an examination result acquired by an examination device that examines the patient (for example, data of an examination result of at least one of visual acuity, axial length of the eye, intraocular pressure, and visual field, etc.).

[0047] Returning to the description of FIG. 2. The CPU 22 sets, as a reference image, one or more medical images 41 to be used as a reference for extracting case data 50 from among the one or more medical images 41 included in the target data 40 acquired in S1 (S2). Although details will be described later, the medical information processing apparatus 20 of the present embodiment extracts, as similar case data, case data 50 including a medical image 51 similar to at least one medical image 41 included in the target data 40 from among the plurality of case data 50 stored in the database. In S2, a medical image 41 for determining the similarity with the medical image 51 in the case data 50 is set as the reference image from among the medical images 41 in the target data 40.

[0048] When setting the reference image in S2, a specific method can be appropriately selected. First, when only one medical image 41 is included in the target data 40 obtained in S1, the CPU 22 sets one medical image 41 in the target data 40 as the reference image. When a plurality of medical images 41 are included in the target data 40, the CPU 22 may set one or a plurality of medical images 41 selected according to the instruction as the reference image based on the selection instruction of the reference image input from the user. Also, when a plurality of medical images 41 are included in the target data 40, the CPU 22 may automatically set one or a plurality of medical images 41 as the reference image in the order of the latest shooting timing.

[0049] Next, the CPU 22 extracts one or a plurality of case data 50 similar to the target data 40 from among the plurality of case data 50 stored in the database as similar case data (S3). Specifically, the CPU 22 extracts case data 50 including a medical image 51 similar to the reference image set in the target data 40 as similar case data.

[0050] As an example, in S3 of the present embodiment, the CPU 22 acquires values indicating the feature amounts of the reference image and each of the plurality of medical images 51 in the case data 50. In the present embodiment, SIFT (Scale Invariant Feature Transform) is adopted as the value indicating the feature amount of the image. However, feature amounts other than SIFT may be acquired. The CPU 22 extracts case data 50 including a medical image 51 having a small difference in feature amount from the reference image as similar case data. For example, the CPU 50 may extract case data 50 including a medical image 51 whose difference in feature amount is equal to or less than a threshold value. Also, the CPU 50 may extract one or a plurality of case data 50 in the order of the smallest difference in feature amount between the reference image and the medical image 51.

[0051] Referring to FIG. 3, an example of a method for extracting similar case data when a plurality of reference images 41A and 41B are set in target data 40 will be described. In the example shown in FIG. 3, among the plurality of medical images 41 included in the target data 40, two medical images are set as reference images 41A and 41B. In this case, the CPU 22 identifies the shooting interval D1 between the plurality of reference images 41A and 41B based on the information on the shooting timing included in the target data 40. Next, the CPU 22 identifies case data 50 that includes a plurality of medical images 51 taken at an interval such that the difference from the shooting interval D1 between the plurality of reference images 41A and 41B is equal to or less than a threshold value among the plurality of case data 50. The CPU 22 extracts, from the identified case data 50, case data 50 that includes a plurality of medical images 41 whose difference in shooting interval from the reference images 41A and 41B is equal to or less than the threshold value and whose similarity to the reference images 41A and 41B is high (that is, the difference in feature amounts is small) as similar case data. Note that the threshold value for comparing the shooting intervals may be set in advance or may be set according to an instruction from the user.

[0052] In the example shown in FIG. 3, the difference between the shooting interval D2 between the two medical images 51A and 51B included in the case data 50A and the shooting interval D1 between the two reference images 41A and 41B is equal to or less than the threshold value. Further, the similarity between the two medical images 51A and 51B and the two reference images 41A and 41B is high (that is, the similarity between the medical image 51A and the reference image 41A and the similarity between the medical image 51B and the reference image 41B are both high). Therefore, the CPU 22 extracts the case data 50A as similar case data. On the other hand, although the similarity between the two medical images 51X and 51Y included in the case data 50B and the two reference images 41A and 41B is high, the difference between the shooting interval D3 between the two medical images 51X and 51Y and the shooting interval D1 between the two reference images 41A and 41B is greater than the threshold value. Therefore, the CPU 22 does not extract the case data 50B as similar case data. By performing the above processing, the extracted similar case data will include the medical images 51A and 51B taken at an interval close to the shooting interval between the plurality of reference images 41A and 41B of the target data 40.

[0053] In addition, when there is one reference image set within the target data 40, the CPU 22 extracts case data 50 including a medical image 51 with a high similarity to the single reference image as similar case data.

[0054] Also, in S3 of the present embodiment, the CPU 22 can extract similar case data from one or more of a plurality of case data groups classified by treatment information. For example, when an instruction to extract similar case data is input from a case data group of a patient who has received a specific treatment, the CPU 22 extracts one or more similar case data from the case data group corresponding to the content of the instructed treatment. Further, the CPU 22 can also extract one or more similar case data from each of the plurality of case data groups. For example, the CPU 22 can also extract similar case data of patients who have received treatment and similar case data of patients who have not received treatment respectively. Also, the CPU 22 can extract a plurality of similar case data with different treatment contents that have been executed.

[0055] Furthermore, in S3 of the present embodiment, the CPU 22 can also extract similar case data by comparing the similarity between a medical image 51 before or after the treatment timing indicated by the treatment information and the reference image of the target data 40 among the plurality of medical images 51 included in each case data 50. For example, when the reference image of the target data 40 is a pre-treatment medical image 41, the CPU 22 compares the similarity between the medical image 51 before the treatment timing and the reference image to extract similar case data. As a result, case data 50 in which the pre-treatment disease state is close to the disease state of the diagnostic subject is appropriately extracted.

[0056] It is also possible to change the method of extracting similar case data. For example, the CPU 22 may input the medical images 41 and 51 into a mathematical model trained by a machine learning algorithm to obtain the similarity of the medical images 41 and 51, and extract similar case data based on the obtained results. Further, when the image quality of the medical image 51 included in the case data 50 is less than the threshold value, the CPU 22 may exclude the case data 50 from the extraction target, or may warn the user that the image quality of the extracted similar case data is low.

[0057] Return to the description of FIG. 2. When similar case data is extracted (S3), the CPU 22 performs a process of matching the time axes of the shooting timings of the medical images for a plurality of data (S5, S6, S7, S9, S10, S13, S14). The plurality of data for which the time axes are to be matched may be the similar case data 50 and the target data 40, or may be a plurality of extracted similar case data 50.

[0058] In the present embodiment, the user can input an instruction to select any one of a method of manually matching the time axis, a method of automatically matching the time axis according to the similarity of the medical images, and a method of automatically matching the time axis according to the treatment timing, by operating the operation unit 25.

[0059] When an instruction to manually match (adjust) the time axes is input (S5: YES), the CPU 22 causes the medical images of a plurality of data (similar case data 50 and target data 40, or a plurality of similar case data 50) to be displayed on the display unit 26 together with the time axes (S6). In the example shown in FIG. 4, the medical images 41A and 41B included in the target data 40 and the medical images 51A, 51B, 51C, and 51D included in the extracted similar case data 50A are displayed on the display unit 26 together with the time axis T. On the time axis T, the imaging timings of the respective medical images are shown. For example, the imaging timing of the medical image 41A is indicated by "A" on the time axis T, and the imaging timing of the medical image 51D is indicated by "d" on the time axis T. Further, in the example shown in FIG. 4, based on the treatment information included in the similar case data 50A, the treatment timing at which the treatment was executed or started is also shown. In the example shown in FIG. 4, the timing of the start of medication coincides with the imaging timing of the medical image 51B.

[0060] The CPU 22 matches the time axes of the imaging timings of the medical images among a plurality of data (in the example shown in FIG. 4, between the target data 40 and the similar case data 50A) according to an instruction input from the user. Further, the CPU 22 changes the imaging timing of each medical image shown on the time axis T according to the result of matching the time axes (S7). Note that the CPU 22 may change the display positions and the like of the plurality of medical images according to the matched time axis together with the imaging timing on the time axis T. For example, the user may input an instruction to match the time axes by inputting, via the operation unit 25, an instruction to slide the imaging timing shown on the time axis T in a direction along the time axis. Further, the user may input an instruction to specify two medical images (in the example shown in FIG. 4, the medical image 41A of the target data 40 and the medical image 51A of the similar case data 50A) whose imaging timings are to be matched among the medical images included in each of the plurality of data. In this case, the CPU 22 matches the time axes of the plurality of data so that the imaging timings of the two specified medical images coincide.

[0061] When an instruction to match the time axes based on the similarity of medical images is input (S9: YES), the CPU 22 compares the similarities of the medical images included in each of the plurality of data (similar case data 50 and target data 40, or a plurality of similar case data 50) among the data, and matches the shooting timings of the medical images with the highest similarity to match the time axes of the plurality of data (S10). In the state where the time axes are matched in S10, the CPU 22 causes the display unit 26 to display both the medical images included in each of the plurality of data and the time axis T indicating the shooting timing of the medical images as medical information (S11). When comparing the medical images of the plurality of data, the disease states in the two medical images with the highest similarity are likely to be approximated. Therefore, by performing the process of S10, the user can more appropriately grasp the transition of the disease based on the medical information.

[0062] When an instruction to match the time axes based on the treatment timing is input (S13: YES), the CPU 22 matches the time axes of the plurality of data by matching the treatment timings among the plurality of data (similar case data 50 and target data 40, or a plurality of similar case data 50) (S14). In the state where the time axes are matched in S14, the CPU 22 causes the display unit 26 to display both the medical images included in each of the plurality of data and the time axis T indicating the shooting timing of the medical images as medical information (S15). By performing the process of S14, the user can compare the plurality of data based on the treatment timing, so that the transition of the disease due to treatment can be appropriately predicted. When the treatment has not yet been executed for the diagnostic subject for whom the target data 40 has been acquired, etc., the time axes may be matched based on the treatment timing assuming that future treatment will be executed for the diagnostic subject. In this case, the user may input the future treatment timing, or the shooting time of the latest medical image 41 may be set as the treatment timing.

[0063] Next, the CPU 22 determines whether an instruction to display the predicted image of the subject to be diagnosed has been input by the user (S17). The predicted image is an image predicted to show the future disease state of the subject to be diagnosed. If an instruction to display the predicted image has been input (S17: YES), the CPU 22 generates a predicted image based on the medical image 41 included in the target data 40 and the medical image 51 included in the similar case data, and causes the display unit 26 to display it (S18).

[0064] With reference to FIG. 5, an example of a method for generating the predicted image 60 will be described. First, the CPU 22 extracts, as the base image 42, the medical image 41 to be used for generating the predicted image from the medical images 41 included in the target data 40 (see FIGS. 3 and 4) for the subject to be diagnosed. The base image 42 used for generating the predicted image is preferably as new an image as possible. Therefore, when the target data 40 includes a plurality of medical images 41, the CPU 22 extracts the medical image 41 with the most recent imaging timing as the base image 42. Next, the CPU 22 extracts, as the reference image 52, among the plurality of medical images 51 included in the similar case data 50 (see FIG. 4), the medical image 51 that was taken at a timing later than the imaging timing of the base image 42 on the matched time axis. In the present embodiment, the reference image 52 is extracted in a state where the time axes of the imaging timings of the target data 40 and the similar case data 50 are matched. Therefore, the reference image 52 for generating the predicted image 60 is more appropriately extracted.

[0065] In the example shown in FIG. 5, the base image 42 includes the diseased part 43, and the reference image 52 also includes the diseased part 53. However, in the reference image 52, the disease has progressed as a result of no treatment being performed on the patient, and the diseased part 53 larger than the diseased part 43 of the base image 42 is captured.

[0066] The CPU 22 generates a predicted image 60 based on the base image 42 and the reference image 52. As an example, in the present embodiment, the CPU 22 generates a diseased part removal image 45 from which information on the diseased part 43 has been removed from the base image 42. Further, the CPU 22 generates a diseased part image 55 from which information other than the diseased part 53 has been removed from the reference image 52. The CPU 22 generates the predicted image 60 based on the diseased part removal image 45 and the diseased part image 55 (for example, by performing synthesis by image processing or the like). As a result, based on the base image 42 and the reference image 52, a predicted image that is predicted to show the future disease state of the subject to be diagnosed is appropriately generated.

[0067] Note that it is also possible to change the specific method for generating the predicted image 60. For example, the predicted image 60 may be generated by using a mathematical model trained by a machine learning algorithm. In this case, for example, the mathematical model may be pre-trained with data of a plurality of medical images so as to output the predicted image 60 by inputting the base image 42 and the reference image 52. Further, at least any one of a mathematical model that inputs the base image 42 and outputs the diseased part removal image 45, a mathematical model that inputs the reference image 52 and outputs the diseased part image 55, and a mathematical model that inputs the diseased part removal image 45 and the diseased part image 55 and outputs the predicted image 60 may be used.

[0068] Returning to the description of FIG. 2. The CPU 22 determines whether an instruction to display a progress transition graph showing the transition of the disease progress in each of a plurality of data (similar case data 50 and target data 40, or a plurality of similar case data 50) has been input (S20). If an instruction to display the progress transition graph has been input (S20: YES), the CPU 22 acquires information on the disease progress shown in each of the plurality of medical images (S21). The CPU 22 generates a progress transition graph showing the transition of the progress in each data with the time axes of the imaging timings among the plurality of data matched, and causes the display unit 26 to display it (S22).

[0069] Referring to FIG. 6, an example of a method for generating a progress transition graph will be described. In the example shown in FIG. 6, in the case data extraction process (see S3, FIG. 2), similar case data of patients who have undergone both surgery and medication (black circle graph), similar case data of patients who have undergone only medication (black square graph), and similar case data of patients who have not received treatment (white circle graph) are extracted from each case data group. In addition, the progress of the subject under examination is shown by the white square graph.

[0070] First, the CPU 22 acquires information on the progress of the disease shown in each of the medical images included in each data. The method for acquiring information on the progress of the disease can be appropriately selected. For example, the CPU 22 may input a medical image into a mathematical model trained by a machine learning algorithm and obtain information on the progress (e.g., the probability of the disease, etc.) by causing the mathematical model to output the progress. In addition, the CPU 22 inputs a medical image into a mathematical model that outputs an analysis result of the disease, and based on information indicating the distribution of the degree of influence (sometimes referred to as an "attention map") affected when the mathematical model outputs the analysis result, the CPU 22 may obtain information on the progress of the disease. In addition, the CPU 22 may perform image processing on each medical image and obtain information on the progress based on at least one of the size and color of the diseased part. Note that the aforementioned attention map may be displayed on the display unit 26 together with the original medical image.

[0071] The CPU 22 creates a progress transition graph showing the progress transition of the disease for each of a plurality of data (in this embodiment, the target data 40 and three similar case data 50) based on the progress information obtained from the medical images. Here, the CPU 22 generates a progress transition graph with the time axes of the imaging timings among the plurality of data matched. In the example shown in FIG. 6, in S13 of FIG. 2, the time axes of the imaging timings are matched so that the treatment timings in the data of the patients who have received treatment match. Therefore, in the example shown in FIG. 6, the user can easily compare the progress transitions of the disease according to the presence or absence of treatment and the treatment content with reference to the treatment timing.

[0072] In the example shown in FIG. 6, among the plurality of medical images 41 included in the target data 40, the imaging timing of the latest medical image 41 is adjusted to the treatment timing of other data as the provisional treatment timing. Therefore, the user can easily predict the transition of the disease when starting treatment urgently by means of the progression transition graph. Also, in the example shown in FIG. 6, among the similar case data of patients for whom treatment has not been performed, the imaging timing of the medical image 51 with the highest similarity to the latest medical image 41 in the target data 40 is adjusted to the imaging timing of the latest medical image 41 in the target data 40.

[0073] Also, when the above-described different type data is included in the similar case data extracted in S3, the CPU 22 causes the display unit 26 to display the different type data together with the medical images included in the similar case data or separately from the medical images. Therefore, the user can more appropriately predict the transition of the disease of the person to be diagnosed by checking the different type data included in the similar case data. Also, even when the different type data regarding the person to be diagnosed has not yet been acquired, the user can check the different type data of patients with similar cases, so that the diagnosis etc. of the person to be diagnosed can be performed more appropriately.

[0074] The technology disclosed in the above embodiment is merely an example. Therefore, it is also possible to change the technology exemplified in the above embodiment. For example, the CPU 22 of the medical information processing apparatus 20 may specify, for each of the plurality of pieces of data, the timing (change timing) at which the change in the feature amounts of the plurality of medical images included in each piece of data becomes equal to or greater than a threshold value. The CPU 22 may match the time axes by matching the change timings in the data. The change timing may be, for example, the timing at which the change in the progress of a disease (the progress of deterioration) becomes equal to or greater than a threshold value, or the timing at which the change in the progress of the cure of a disease becomes equal to or greater than a threshold value. Further, the CPU 22 may extract similar case data by comparing the similarity between the medical images before or after the change timing among the plurality of medical images 51 included in the case data 50 and the medical image 41 of the target data 40.

[0075] The process of acquiring the target data 40 in S1 of FIG. 2 is an example of the "target data acquisition step". The process of extracting similar case data in S3 of FIG. 2 is an example of the "similar case data extraction step". The processes of matching the time axes of the imaging timings in S7, S10, and S14 of FIG. 2 are examples of the "time axis matching step". The processes of outputting medical information in S7, S11, S15, S18, and S22 of FIG. 2 are examples of the "medical information output step". The process of generating a predicted image in S18 of FIG. 2 is an example of the "predicted image generation step". The process of acquiring information on the degree of progression in S21 of FIG. 2 is an example of the "degree of progression acquisition step". The process of generating a degree-of-progression transition graph in S22 of FIG. 2 is an example of the "graph generation step".

Explanation of Signs

[0076] 10 Server 13 Storage device 20 Medical information processing apparatus 22 CPU 23 Storage device 26 Display unit 40 Target data 41 Medical image 50 Case data 51 Medical image 60 Prediction Image

Claims

1. A medical information processing program executed in a medical information processing apparatus that outputs medical information useful for diagnosis based on case data similar to the data of a subject to be diagnosed among a plurality of case data, each of the plurality of case data is stored in a database in a state including data of a plurality of medical images taken at different timings, information on the imaging timing of each medical image, and treatment information regarding treatment executed on the patient, the treatment information included in the case data includes information indicating the treatment timing at which the treatment was executed or started, by the medical information processing program being executed by the control unit of the medical information processing apparatus, a target data acquisition step of acquiring target data including data of at least one medical image regarding the subject to be diagnosed and information on the imaging timing of the medical image; a similar case data extraction step of extracting, as similar case data, the case data including a medical image similar to at least one medical image included in the target data from the plurality of case data; a time axis matching step of matching the time axis of the imaging timing of the medical image between the similar case data and the target data; a medical information output step of outputting medical information based on the similar case data by causing a display unit to display the medical image included in the target data and the plurality of medical images included in the similar case data, and displaying the imaging timing of the medical image of the target data and the imaging timing and treatment timing of the plurality of medical images of the similar case data on the time axis in a state where the time axes are matched; A medical information processing program, characterized in that it is executed by the medical information processing apparatus.

2. The medical information processing program according to Claim 1, in the similar case data extraction step, the control unit extracts the similar case data from among one or a plurality of case data groups classified within the plurality of case data according to at least any one of the presence or absence of treatment indicated by the treatment information and the content of the executed treatment. A medical information processing program characterized by this.

3. The medical information processing program according to Claim 1 or 2, In the time axis matching step, the control unit matches the time axes by matching the treatment timings among a plurality of data. A medical information processing program characterized by this.

4. A medical information processing program according to any one of Claims 1 to 3, wherein the control unit in the medical image included in the case data extracted in the similar case data extraction step, at the matched time axis, based on the medical image taken after the same timing as the imaging timing of the medical image included in the target data and the medical image included in the target data, further executes a predicted image generation step of generating a predicted image of the subject to be diagnosed, In the medical information output step, the control unit causes the display unit to display the predicted image generated with the time axes matched as the medical information. A medical information processing program characterized by this.

5. A medical information processing program according to any one of Claims 1 to 4, wherein the control unit executes a progress degree acquisition step of acquiring information on the progress degree of the disease shown in each of a plurality of medical images, and a graph generation step of generating a progress degree transition graph showing the transition of the progress degree in each of the similar case data and the target data, or in each of the plurality of similar case data, with the time axes of the plurality of data matched, and further executes In the medical information output step, the progress degree transition graph generated with the time axes matched is caused to be displayed on the display unit as the medical information. A medical information processing program characterized by this.

6. A medical information processing program according to any one of Claims 1 to 5, wherein in the time axis matching step, the control unit matches the time axes of the imaging timings of medical images among a plurality of data in response to an instruction input from a user. A medical information processing program characterized by this.

7. A medical information processing apparatus that outputs medical information useful for diagnosis based on case data similar to the data of a subject to be diagnosed among a plurality of case data, each of the plurality of case data is stored in a database in a state including data of a plurality of medical images taken at different timings, information on the imaging timing of each medical image, and treatment information regarding the treatment performed on the patient. The treatment information included in the case data includes information indicating the treatment timing at which the treatment was executed or started. The control unit of the medical information processing apparatus A target data acquisition step of acquiring target data including data of at least one medical image regarding the diagnosis target person and information on the imaging timing of the medical image; A similar case data extraction step of extracting, as similar case data, the case data including a medical image similar to at least one medical image included in the target data from the plurality of case data; A time axis matching step of matching the time axes of the imaging timings of the medical images among the similar case data, the target data, and at least one of the plurality of similar case data; A medical information output step of causing a display unit to display the medical image included in the target data and the plurality of medical images included in the similar case data, and displaying the imaging timing of the medical image of the target data, the imaging timings and treatment timings of the plurality of medical images of the similar case data on the time axis in a state where the time axes are matched, thereby outputting medical information based on the similar case data; A medical information processing apparatus characterized by executing the above.

Images