Information processing device, computer program, and information processing method

Abstract

This study aims to clarify an information processing device, computer program, information processing method, and rule generation method that can implement rules for determining the necessity of final image interpretation in a way that reduces the burden on physicians. [Solution] The information processing device includes a first acquisition unit that acquires a determination result of whether or not a final image interpretation is necessary based on a medical image, a second acquisition unit that acquires a final image interpretation result based on a medical image for which a final image interpretation has been determined to be necessary, and a generation unit that generates a necessity determination rule for determining whether or not a final image interpretation is necessary based on the necessity determination result and the final image interpretation result.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, a computer program, an information processing method, and a rule generation method.

Background Art

[0002] In recent years in the medical field, with the progress of radiation diagnostic equipment, the amount of image diagnostic information has increased exponentially, and the number of examinations has also increased. On the other hand, the current situation is that the number of specialists who perform radiation diagnosis professionally is small. For this reason, medical images taken at medical institutions such as clinics or health examination institutions are not only read by specialists at the medical institutions or health examination institutions, but also transmitted to another facility where there are specialists using a communication network, and remote reading may be performed by the specialists at that facility.

[0003] Patent Document 1 discloses a remote reading system in which a medical image is transmitted from a requesting facility to a reading center, and at the reading center, primary reading and secondary reading are performed by two radiologists. If there are differences in the reading results of both, final reading is performed by another radiologist.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, if a radiologist is fatigued or needs to interpret a large number of images in a short time, their interpretation ability may decline, potentially leading to oversights and unnecessary interpretations being requested from the final radiologist. While using image analysis results to support radiologists is an option, it could lead to misdiagnosis and actually increase the burden on the final radiologist. Furthermore, manually deciding whether or not to send an image for final interpretation would increase the workload at the radiology facility.

[0006] This disclosure is made in light of the circumstances described above, and reveals an information processing device, computer program, information processing method, and rule generation method that can realize a final interpretation determination rule that can reduce the burden on physicians and the workload at image interpretation facilities. [Means for solving the problem]

[0007] The present invention includes multiple means for solving the above-mentioned problems, but to give one example, the information processing device comprises a first acquisition unit that acquires a determination result of whether or not a final image interpretation is necessary based on a medical image, a second acquisition unit that acquires a final image interpretation result based on a medical image for which a final image interpretation has been determined to be necessary, and a generation unit that generates a necessity determination rule for determining whether or not a final image interpretation is necessary based on the necessity determination result and the final image interpretation result. [Effects of the Invention]

[0008] According to this disclosure, it is possible to implement rules for determining whether final image interpretation is necessary, thereby reducing the burden on physicians and the workload at image interpretation facilities. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of the configuration of the remote image interpretation system of this embodiment. [Figure 2] This figure shows an example of a server configuration. [Figure 3] This figure shows an example of remote image interpretation processing performed by a server. [Figure 4] This figure shows an example of the process by which the server generates rules for determining necessity. [Figure 5] This figure shows the first example of a group of medical images using the previous necessity determination rules. [Figure 6] This figure shows the first example of generation conditions for generating necessity determination rules. [Figure 7] This figure shows the first example of a set of medical images using the newly generated necessity determination rule. [Figure 8] This figure shows a second example of generation conditions for generating necessity determination rules. [Figure 9] This figure shows a third example of generation conditions for generating necessity determination rules. [Figure 10] This figure shows a second example of a group of medical images using the previous necessity determination rules. [Figure 11] This figure shows a second example of a set of medical images using the newly generated necessity determination rule. [Figure 12] This figure shows an example of the server's process for updating the necessity determination rule. [Figure 13] This diagram shows an example of criteria for determining whether the impact on the final image interpretation is within an acceptable range. [Figure 14] This figure shows an example of how the server evaluates the necessity determination rules. [Modes for carrying out the invention]

[0010] Embodiments of this disclosure will be described below with reference to the drawings. Figure 1 is a diagram showing an example of the configuration of the remote image interpretation system of this embodiment. The remote image interpretation system includes a server 50 as an information processing device. A first image interpretation terminal 10, a second image interpretation terminal 20, a final image interpretation terminal 30, and a remote image interpretation DB 40 are connected to the server 50. The first image interpretation terminal 10, the second image interpretation terminal 20, and the final image interpretation terminal 30 can be configured as, for example, personal computers. The server 50, the first image interpretation terminal 10, the second image interpretation terminal 20, the final image interpretation terminal 30, and the remote image interpretation DB 40 are installed, for example, within the facilities of a remote image interpretation service company. The first image interpretation terminal 10, the second image interpretation terminal 20, and the final image interpretation terminal 30 are terminals used by the first radiologist, the second radiologist, and the final radiologist, respectively, when interpreting images. The first radiologist, the second radiologist, and the final radiologist are each different radiologists.

[0011] Server 50 is connected to communication network 1. Terminal devices 101, 201, and 301 of the health checkup facilities 100, 200, and 300 are connected to communication network 1. These terminal devices 101, 201, and 301 can be composed of, for example, personal computers.

[0012] The terminal device 101 of the health checkup facility 100 can send and receive information with the terminal device 150 of the person being checked, for example, via email. Although the terminal device 150 of the person being checked is only illustrated for the health checkup facility 100 for convenience, the terminal devices 201 and 301 of the other health checkup facilities 200 and 300 can similarly send and receive information with the terminal devices of the person being checked, via email, etc. The terminal device 150 can consist of a smartphone, tablet, personal computer, etc.

[0013] The health examination institutions 100, 200, and 300 are, for example, institutions where members belonging to a health insurance association receive health examinations. They transmit medical images taken with an X-ray imaging device or the like for a large number of examinees to the server 50 through the terminal devices 101, 201, and 301. Also, although not shown in the figure, similarly, for medical institutions such as hospitals and clinics without a radiologist, medical images taken with an X-ray imaging device or the like for patients can be transmitted to the server 50 through the terminal device.

[0014] The remote radiology service company has a professional radiologist read the medical images requested from the health examination institutions 100, 200, and 300 and medical institutions, and reports the final result as the remote radiology service company to the health examination institutions 100, 200, and 300 and medical institutions. The final result describes the findings regarding the read medical images.

[0015] In this specification, medical images include images obtained by imaging with inspection devices such as X-ray imaging devices, ultrasonic inspection devices, CT (Computed Tomography) devices, MRI (Magnetic Resonance Imaging) devices, etc. Reading includes a professional radiologist looking at the medical images obtained by the inspection, judging the medical images, and creating the presence or absence of findings and the content of the findings as a report. Findings are the results of reading, and indicate abnormal results such as nodules and infiltration shadows in the medical images to be read, as well as opinions on the results. Also, findings may be not only abnormal results, but also shown as "no findings" in the report even if they are normal. Note that findings include not only those by radiologists, but also results by image analysis processing by computers such as CAD (Computer Aided Detection: image processing software). Medical images may be stored as DICOM (Digital Imaging and Communications in Medicine) format data. DICOM data includes image information and tag information such as patient name and imaging site. Findings may be written as tag information in the DICOM data in which the medical image is stored, or may be stored as separate data from the DICOM data including the medical image. For example, findings by a radiologist may include information such as the presence or absence and content and name of the findings, and may be stored as plain text or encoded report data. Findings by image analysis processing may be stored as DICOM-SR data in which the presence or absence and content and name of the findings are encoded, or as DICOM-SC data imaged like a heat map.

[0016] FIG. 2 is a diagram showing an example of the configuration of server 50. Server 50 includes a control unit 51 that controls the entire server 50, a communication unit 52, an interface unit 53, an image processing unit 54, a recording medium reading unit 55, a memory 56, a generation unit 57, an update unit 58, and a storage unit 59.

[0017] The control unit 51 is configured by incorporating all or part of the required number of CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), GPGPUs (General-purpose computing on graphics processing units), TPUs (Tensor Processing Units), etc. The control unit 51 has the functions of a first acquisition unit, a second acquisition unit, a third acquisition unit, and a fourth acquisition unit.

[0018] Memory 56 is composed of semiconductor memory such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), and flash memory.

[0019] The communication unit 52, for example, is equipped with a communication module and has a communication function with terminal devices 101, 201, and 301 via the communication network 1. The communication unit 52 receives medical images to be read and requests for reading from terminal devices 101, 201, and 301 via the communication network 1. The communication unit 52 transmits the final results regarding the medical images to be read to terminal devices 101, 201, and 301 via the communication network 1.

[0020] The interface unit 53 provides interface functionality between the first image interpretation terminal 10, the second image interpretation terminal 20, the final image interpretation terminal 30, and the remote image interpretation database 40. The server 50 can transmit and receive information to and from the first image interpretation terminal 10, the second image interpretation terminal 20, and the final image interpretation terminal 30 through the interface unit 53. The server 50 can access the remote image interpretation database 40 through the interface unit 53.

[0021] The recording medium reading unit 55 can be configured, for example, as an optical disc drive, and can read a computer program (program product) recorded on the recording medium 551 (for example, an optically readable disc storage medium such as a CD-ROM) and store it in the storage unit 59. The computer program 61 is loaded into the memory 56 and executed by the control unit 51. Alternatively, the computer program 61 may be downloaded from an external device via the communication unit 52 and stored in the storage unit 59.

[0022] The storage unit 59 can be configured, for example, as a hard disk or semiconductor memory, and can store the necessary information. The storage unit 59 can store judgment rules (rules for determining whether final image interpretation is necessary) 60 and computer programs 61.

[0023] The image processing unit 54 can be composed of a CPU, MPU, GPU, GPGPU, TPU, DSP (Digital Signal Processors), or FPGA (Field-Programmable Gate Arrays), etc. The image processing unit 54 performs image analysis on the medical image to be read and outputs the presence or absence of findings and the content of those findings, similar to those read by a radiologist. The image processing unit 54 may be one or more image processing modules, or it may be a learning model generated by machine learning.

[0024] A learning model comprises an input layer, an intermediate layer, and an output layer, and can be constructed, for example, as a Convolutional Neural Network (CNN). The intermediate layer comprises multiple convolutional layers, multiple pooling layers, and a fully connected layer. When a medical image is input, the learning model can output the presence or absence of findings and the content of those findings. The findings can be pre-encoded. For example, if there are 10 types of findings, the output layer can have 10 output nodes, and each output node can classify the findings.

[0025] The generation unit 57 has the function of generating rules for determining whether or not final image interpretation is necessary (hereinafter also referred to as "necessity determination rules"). The method for generating the necessity determination rules will be described later.

[0026] The update unit 58 has a function to determine whether to update the necessity determination rule newly generated by the generation unit 57 in place of the previous necessity determination rule, and if it determines to update, it has a function to update the necessity determination rule. The method for updating the necessity determination rule will be described later.

[0027] The Remote Image Interpretation DB40 stores the medical images to be interpreted, interpretation results based on the interpreting physician (including the presence or absence of findings, the content of the findings, etc.), results from image analysis processing (including the presence or absence of findings, the content of the findings, etc.), the determination of whether a final interpretation is necessary, the interpretation results based on the final interpretation (including the presence or absence of findings, the content of the findings, etc.), and the final results as a remote image interpretation service company. The Remote Image Interpretation DB40 has accumulated medical images from the past, interpretation results, results from image analysis processing, determination of whether a final interpretation is necessary, interpretation results based on the final interpretation, and the final results.

[0028] Next, the processing procedure of server 50 will be described. The processing of server 50 can be implemented by computer program 61.

[0029] Figure 3 shows an example of remote image interpretation processing by server 50. For convenience, only terminal device 101 of health checkup institution 100 is shown below. The same applies to other health checkup institutions 200, 300 and medical institutions. Server 50 receives the medical images to be interpreted and the interpretation request from terminal device 101 (S1). The medical images to be interpreted can be received in batches, for example, in lot units. One lot may contain, for example, several tens to several hundred medical images.

[0030] Server 50 temporarily stores the received medical image to be read in the remote image reading DB 40 (S2), performs image analysis processing on the medical image using the image processing unit 54, and saves the image analysis results to the remote image reading DB 40 (S3). The image analysis results include the presence or absence of findings based on the medical image and the content of those findings.

[0031] Server 50 transmits the medical image to be read to the first reading terminal 10 (S4). The first reading terminal 10 performs the initial reading (first interpretation) by the radiologist (first radiologist) (S5). Server 50 receives the initial reading result from the first reading terminal 10 (S6) and saves the received initial reading result to the remote reading DB 40 (S7). The initial reading result includes whether or not there are findings based on the medical image by the radiologist (first radiologist) and the content of those findings.

[0032] In the case of double image interpretation, the server 50 performs the following processing: The server 50 transmits the medical image to be interpreted to the second interpretation terminal 20 (S8). The second interpretation terminal 20 performs a second interpretation (secondary interpretation) by the radiologist (second radiologist) (S9). The server 50 receives the second interpretation result from the second interpretation terminal 20 (S10) and saves the received second interpretation result to the remote interpretation DB 40 (S11). The second interpretation result includes whether or not there are findings based on the medical image by the radiologist (second radiologist) and the content of those findings. In the case of single image interpretation, the processing in steps S8 to S11 described above is not performed.

[0033] Server 50 determines whether a final image interpretation is necessary (S12). The determination of whether a final image interpretation is necessary is made based on the necessity determination rules. Server 50 saves the necessity determination result to the remote image interpretation DB 40 (S13). If it is determined that a final image interpretation is necessary, the following processing is performed. Specifically, Server 50 transmits the medical image to be interpreted, along with the associated first and second image interpretation results, image analysis processing results, and information on the necessity determination result to the final image interpretation terminal 30 (S14). At the final image interpretation terminal 30, the radiologist (the final radiologist) performs the final image interpretation (S15). Server 50 receives the final image interpretation result from the final image interpretation terminal 30 (S16) and saves the received final image interpretation result to the remote image interpretation DB 40 (S17). The final image interpretation result includes the presence or absence of findings based on the medical image by the radiologist (the final radiologist) and the content of those findings. If it is determined that a final image interpretation is not necessary, the processes described in steps S14 to S17 above will not be performed.

[0034] Server 50 generates the final result for the medical image to be interpreted (S18), sends the generated final result to terminal device 101 (S19), and terminates processing.

[0035] As described above, the server 50 acquires the medical image to be read via the communication unit 52, determines whether there are any findings based on the interpretation of the acquired medical image, determines whether there are any findings based on the image analysis processing of the acquired medical image, and determines whether a final interpretation of the acquired medical image is necessary based on the results of each determination.

[0036] This means that not only the radiologist's interpretation but also the presence or absence of findings from image analysis processing (including estimation by a learning model) is considered, thus complementing the radiologist's interpretation and reducing the likelihood of overlooking findings during interpretation.

[0037] Server 50 can generate (obtain) a final result indicating the presence or absence of findings based on the interpretation of the medical image being interpreted, according to the final interpretation result. The final result includes the presence or absence of findings and the content of the findings (for example, the location and size of lesions such as nodules). In cases where a final interpretation is not required, the final result is generated based on both the first and second interpretation results (both of which match regarding the presence or absence of findings) in the case of double interpretation, and based on the first interpretation result in the case of single interpretation.

[0038] When terminal device 101 receives the final results from server 50, the person in charge at the health checkup implementing institution 100 may create a health checkup result based on the final results and send (provide) the created health checkup result to the terminal device 150 of the person being checked. The health checkup result may also be mailed to the person being checked at home or at work. Final image interpretation refers to the final image interpretation in relation to this information processing device, and further image interpretations may be performed after the final interpretation.

[0039] Next, we will explain how the generation unit 57 generates the necessity determination rules.

[0040] Figure 4 shows an example of the process by which the server 50 generates necessity determination rules. For convenience, the control unit 51 will be used as the main unit of the process below. The control unit 51 accesses the remote image interpretation DB 40 to obtain the image analysis processing results of the medical image group, the necessity determination results for the final interpretation, and the final interpretation results (S21). The image analysis processing results include the presence or absence of findings and the content of the findings as determined by the image analysis processing by the image processing unit 54. The necessity determination results for the final interpretation are the determination results made according to the previous (or current) necessity determination rules. The final interpretation results are the interpretation results made by the final interpreting physician.

[0041] The control unit 51 identifies the medical images that have undergone final interpretation from the acquired medical image group (S22), and identifies the medical images with a positive final interpretation result as the first group (S23). A positive final interpretation result means, for example, that the image analysis processing result shows findings, and the final interpretation result also shows findings. The control unit 51 identifies the medical images with a false positive final interpretation result as the second group (S24). A false positive final interpretation result means, for example, that the image analysis processing result shows findings, but the final interpretation result shows no findings.

[0042] The control unit 51 generates a rule for determining whether a final image interpretation is necessary based on the medical images of the first and second groups (S25), and then terminates the process. Specifically, the control unit 51 obtains detection results from image analysis processing based on the medical images to be interpreted, obtains a determination result for determining whether a final image interpretation is necessary based on the medical images, and obtains the final image interpretation result based on the medical images for which a final image interpretation is deemed necessary. The generation unit 57 can generate a rule for determining whether a final image interpretation is necessary based on the obtained determination results and final image interpretation results.

[0043] More specifically, the generation unit 57 functions as a identification unit and can identify a first group of results (positive) in which the final image interpretation result is positive and the detection result by image analysis processing is positive, and identify a second group of results (false positive) in which the final image interpretation result is negative and the detection result by image analysis processing is positive, and generate a necessity determination rule based on the identified result (for example, the boundary between positive and false positive).

[0044] Next, we will explain a specific example of how to generate rules for determining necessity.

[0045] Figure 5 shows the first example of a group of medical images using the previous necessity determination rules. The medical image group shown in Figure 5 shows, for medical images 1 to 10, the presence or absence of findings in the first reading, the presence or absence of findings in the second reading, the detection results from image analysis processing (presence or absence of findings, and size of the detected object), the necessity determination result of the final reading based on the current necessity determination rules, and the presence or absence of findings in the final reading. Medical images 3, 6 to 9 were determined to require a final reading.

[0046] Medical image 3 shows no findings in the first interpretation, no findings in the second interpretation, findings in the image analysis processing result with a size of 3.5 mm, and no findings in the final interpretation. Since there are findings in the image analysis processing result and no findings in the final interpretation, it can be classified as a false positive.

[0047] Medical image 6 shows no findings in the first and second interpretations, but a finding with a size of 7.5 mm in the image analysis processing results, and a finding in the final interpretation. Since there is a finding in both the image analysis processing results and the final interpretation, it can be classified as positive.

[0048] Medical image 7 shows no findings in the first reading, no findings in the second reading, findings in the image analysis processing, a size of 4.2 mm, and no findings in the final reading (false positive).

[0049] Medical image 8 shows no findings in the first reading, no findings in the second reading, findings in the image analysis processing, a size of 6.2 mm, and a final reading result of findings (positive).

[0050] Medical image 9 shows no findings in the first reading, no findings in the second reading, findings in the image analysis processing, a size of 3.8 mm, and no findings in the final reading (false positive).

[0051] Figure 6 shows the first example of generation conditions for generating necessity determination rules. In the example in Figure 5, the detection size is included in the detection results from the image analysis process, so we focus on the boundary between positive and false positives in terms of detection size. In Figure 5, when we separate the medical images that underwent final interpretation into positive and false positive medical images and focus on the detection size, as shown in Figure 6, the detection sizes of the false positive medical images (group 2) are 3.5 mm, 3.8 mm, and 4.2 mm. Also, the detection sizes of the positive medical images (group 1) are 6.2 mm and 7.5 mm. It can be seen that the boundary between false positives and positives is, for example, a detection size of 6.2 mm.

[0052] Figure 7 shows the first example of a set of medical images using the newly generated necessity determination rule. The newly generated necessity determination rule takes into account the boundary between positive and false positives as illustrated in Figure 6. Specifically, the necessity determination rule can be generated by using the condition "detection size of image analysis processing is 6.2 mm or larger," and determining that final interpretation is necessary if this condition is met, and that final interpretation is unnecessary if this condition is not met.

[0053] For medical images 1-10, the same as in Figure 5, the generated necessity determination rule differs from the one in Figure 5. For medical images 3, 7, and 9, it is determined that final interpretation is not necessary. However, for medical images 6 and 8, the generated necessity determination rule, similar to the one in Figure 5, determines that final interpretation is necessary.

[0054] As described above, in the previous necessity determination rule shown in Figure 5, for six medical images in which the image analysis processing detected findings (including both positive and false positives), the generated necessity determination rule shown in Figure 6 resulted in only two medical images being determined to require final interpretation, thus reducing the proportion. In other words, cases determined to be false positives can be avoided from being requested for final interpretation. Thus, the generation unit 57 can generate necessity determination rules such that the proportion of medical images in which the detection result indicates findings that are determined to require final interpretation is reduced.

[0055] The above configuration enables the creation of a final interpretation determination rule that reduces the burden on physicians by preventing unnecessary interpretation requests to the final interpreter. Furthermore, since the determination rule for whether or not to send an image for final interpretation does not need to be considered manually and the determination rule can be implemented semi-automatically, the workload at the interpretation facility can be reduced. In addition, for medical images that were initially judged to have findings in the image analysis process but ultimately turned out to be false positives, it becomes possible to determine that final interpretation is unnecessary, thereby improving the accuracy of the detection results from the image analysis process.

[0056] Furthermore, in the previous necessity determination rule shown in Figure 5, there are two medical images (medical images 6 and 8) whose final interpretation result shows findings. However, in the generated necessity determination rule shown in Figure 6, there are two medical images (medical images 6 and 8) whose final interpretation result shows findings, and the number of medical images with findings in the final interpretation result is not reduced. In this way, the generation unit 57 can generate necessity determination rules in a way that does not reduce the number of medical images with findings in the final interpretation result.

[0057] Figure 8 shows a second example of generation conditions for generating necessity determination rules. In the first example shown in Figure 6, the boundary between positive and false positives was considered by focusing on the detection size. In Figure 8A, the boundary between positive and false positives is considered from the perspective of pixel values ​​(e.g., number of pixels) in the detection results obtained by image analysis processing. For example, medical images can be divided into positive and false positives by focusing on the number of pixels (number of pixels) that have a pixel value of a predetermined threshold or higher among the pixel values ​​of the region related to the content of the findings. In the example in Figure 8A, the number of pixels in the false positive medical images (group 2) is 20, 41, and 49. The number of pixels in the positive medical images (group 1) is 55 and 62. It can be seen that the boundary between false positives and positives is, for example, a number of pixels of 55. In this case, the necessity determination rule can be generated using the condition "the number of pixels in the image analysis processing result is 55 or more," and if this condition is met, it is determined that final interpretation is necessary, and if this condition is not met, it is determined that final interpretation is unnecessary.

[0058] Figure 8B focuses on the boundary between positive and false positives in terms of shape (e.g., circularity) in the detection results obtained by image analysis processing. Circularity is represented by a numerical value in the range of 0 to 1, with smaller values ​​indicating more complex shapes. For example, by focusing on the circularity of the shape of the region related to the findings, medical images can be divided into positive and false positives. In the example in Figure 8B, the circularity of the false positive medical images (Group 2) is 0.75 and 0.79. The circularity of the positive medical images (Group 1) is 0.85, 0.9, and 0.92. It can be seen that the boundary between false positives and positives is, for example, a circularity of 0.85. In this case, a necessity determination rule can be generated using the condition "the circularity of the image analysis processing result is 0.85 or higher." If this condition is met, it is determined that final interpretation is necessary; if this condition is not met, it is determined that final interpretation is unnecessary.

[0059] As described above, the generation unit 57 may generate a necessity determination rule using at least one of the detection size, pixel value, and shape indicated by the detection result from the image analysis processing as a generation condition.

[0060] Figure 9 shows a third example of generation conditions for generating necessity determination rules. In Figure 9A, we focus on the boundary between positive and false positives in terms of the malignancy of findings obtained through image analysis processing. Malignancy is expressed as a numerical value in the range of 0 to 1, with the higher the value, the more malignant the finding. Malignancy can be calculated by comprehensively evaluating, for example, the shape and pixel values ​​of the region related to the content of the finding. In the example in Figure 9A, the malignancy values ​​of the false positive medical images (Group 2) are 0.45, 0.5, and 0.6. The malignancy values ​​of the positive medical images (Group 1) are 0.7 and 0.9. It can be seen that the boundary between false positives and positives is, for example, a malignancy value of 0.7. In this case, the necessity determination rule can be generated by using the condition "the malignancy level of the image analysis processing result is 0.7 or higher." If this condition is met, it is determined that a final image interpretation is necessary, and if this condition is not met, it is determined that a final image interpretation is unnecessary.

[0061] Figure 9B focuses on the boundary between positive and false positives from the perspective of the reliability of the image analysis processing. If the image processing unit 54 is a learning model generated by machine learning, the reliability (accuracy) of the output data output by the learning model can be used. In the example in Figure 9B, the reliability of the false positive medical images (group 2) is 68% and 70%. The reliability of the positive medical images (group 1) is 80%, 85%, and 90%. It can be seen that the boundary between false positives and positives is, for example, a reliability of 80%. In this case, the necessity determination rule can be generated by using the condition "the reliability of the image analysis processing result is 80% or higher," and if this condition is met, it is determined that final interpretation is necessary, and if this condition is not met, it is determined that final interpretation is unnecessary.

[0062] As described above, the generation unit 57 may generate necessity judgment rules using the degree of malignancy of the findings indicated by the detection results from the image analysis processing as a generation condition. Alternatively, the generation unit 57 may generate necessity judgment rules using the confidence level of the detection results output by the learning model (image processing unit 54) as a generation condition. The necessity judgment rules are not limited to a configuration that generates them using a single feature quantity, such as the detection size, pixel value, and shape (circularity) indicated by the detection results from the image analysis processing, but may be generated using multiple features. For example, a classifier may be generated using an SVM (Support Vector Machine) with three features (size, pixel value, and circularity) as input, or other methods that utilize multiple features.

[0063] Figure 10 shows a second example of a group of medical images using the previous necessity determination rules. The medical image group shown in Figure 10 shows, for medical images 1 to 10, the presence or absence of findings in the first interpretation, the presence or absence of findings in the second interpretation, the presence or absence of findings from image analysis processing, the necessity determination result of the final interpretation based on the current necessity determination rules, and the presence or absence of findings in the final interpretation. Medical images 2 to 3 and 6 to 10 were determined to require a final interpretation.

[0064] Medical image 2 shows no findings in the first interpretation, findings in the second interpretation, and no findings in the image analysis processing results; therefore, a final interpretation is not necessary.

[0065] Medical image 3 shows no findings in the first interpretation, no findings in the second interpretation, findings in the image analysis processing, and no findings in the final interpretation. Since there are findings in the image analysis processing and no findings in the final interpretation, it can be classified as a false positive.

[0066] Medical image 6 shows findings in the first interpretation, no findings in the second interpretation, findings in the image analysis processing, and findings in the final interpretation. Since the image analysis processing results and the final interpretation result both show findings, it can be classified as positive.

[0067] Medical image 7 shows no findings in the first interpretation, no findings in the second interpretation, findings in the image analysis processing, and no findings in the final interpretation (false positive).

[0068] Medical image 8 shows no findings in the first interpretation, no findings in the second interpretation, findings in the image analysis processing, and no findings in the final interpretation (false positive).

[0069] Medical image 9 shows no findings in the first interpretation, no findings in the second interpretation, findings in the image analysis processing, and no findings in the final interpretation (false positive).

[0070] Medical image 10 shows findings in the first reading, no findings in the second reading, findings in the image analysis processing, and findings in the final reading (positive).

[0071] In the example in Figure 10, the presence or absence of findings is included for each of the first reading, second reading, image analysis processing, and final reading. Therefore, we will focus on the boundary between positive and false positives in terms of the presence or absence of findings. In Figure 10, when we separate the medical images that underwent the final reading into positive and false positive medical images and focus on the presence or absence of findings, we find that the number of findings is 1 in the false positive medical images and 2 in the positive medical images. It can be seen that the boundary between false positives and positives is, for example, somewhere between the number of findings being 2 and 1.

[0072] Figure 11 shows a second example of a group of medical images using the newly generated necessity determination rule. The newly generated necessity determination rule takes into account the boundary between positive and false positives as illustrated in Figure 10. Specifically, the necessity determination rule can be generated by using the condition "two or more findings present," determining that a final interpretation is necessary if this condition is met, and that a final interpretation is unnecessary if this condition is not met.

[0073] The generation unit 57 can generate necessity determination rules using the number of findings related to the image interpretation results and image analysis processing results as generation conditions.

[0074] For medical images 1-10, the same as in Figure 10, the generated necessity determination rule differs from the one in Figure 10. For medical images 2-3 and 7-9, it is determined that final interpretation is not required. However, for medical images 6 and 10, the generated necessity determination rule, similar to the one in Figure 10, determines that final interpretation is required.

[0075] As described above, in the previous necessity determination rule shown in Figure 10, out of seven medical images (medical images 3, 5-10) where the detection result of the image analysis process showed findings (including both positive and false positives), in the generated necessity determination rule shown in Figure 11, only two medical images (medical images 6, 10) were determined to require final interpretation, and the proportion of these two images was reduced. In other words, medical images determined to be false positives can be avoided from being requested for final interpretation. Thus, the generation unit 57 can generate necessity determination rules such that the proportion of medical images with findings that are determined to require final interpretation is reduced.

[0076] The above configuration enables the implementation of a final interpretation determination rule that reduces the burden on physicians by suppressing unnecessary interpretation requests to the final radiologist. Furthermore, for medical images that were initially detected as having findings but ultimately turned out to be false positives, it becomes possible to determine that final interpretation is unnecessary, thereby improving the accuracy of the detection results from image analysis.

[0077] Furthermore, in the previous necessity determination rule shown in Figure 10, there are two medical images (medical images 6 and 10) whose final interpretation result shows findings. However, in the generated necessity determination rule shown in Figure 11, there are two medical images (medical images 6 and 10) whose final interpretation result shows findings, and the number of medical images with findings in the final interpretation result is not reduced. In this way, the generation unit 57 can generate necessity determination rules in a way that does not reduce the number of medical images with findings in the final interpretation result.

[0078] The generation unit 57 may generate necessity determination rules using the name of the finding, the location of the finding, or the normal / abnormal determination category indicated by the finding as generation conditions related to the image interpretation result and the image analysis processing result. The name of the finding can be, for example, nodule. Examples of determination categories include the determination categories specified in each medical image such as health checkups and medical examinations, and there are categories A, B, C, D1, D2, and E. Category A is "no abnormality", category B is "mild abnormality", category C is "requires follow-up observation", category D1 is "requires treatment", category D2 is "requires further examination", and category E is "currently undergoing treatment". Categories A to C are judged as normal, and categories D1 to E are judged as abnormal.

[0079] Regarding the names of findings, for example, a condition such as "two or more different names of findings" can be used in the necessity determination rule. If this condition is met, the rule can be determined to require a final reading; if this condition is not met, the rule can be determined to require a final reading.

[0080] Regarding the location of findings, for example, a condition such as "two or more findings in different locations" can be used in the necessity determination rule. If this condition is met, the rule can be determined to require a final reading; if this condition is not met, the rule can be determined to require a final reading.

[0081] Regarding the determination categories, for example, a determination rule can be generated that uses the condition "one or more findings indicating an abnormality," and if this condition is met, it is determined that a final interpretation is necessary; if this condition is not met, it is determined that a final interpretation is unnecessary. Note that the conditions applied to the determination rule are not limited to the example above.

[0082] Next, we will explain the method for determining whether or not to update the previous necessity determination rules with the necessity determination rules generated by the generation unit 57 and put them into operation.

[0083] Figure 12 shows an example of the update determination process for the necessity determination rule by the server 50. The control unit 51 obtains the necessity determination rule for the final image interpretation generated by the generation unit 57 (S31), and obtains the image interpretation results based on past medical images (e.g., the first image interpretation result, or the first and second image interpretation results), the image analysis processing results, the necessity determination result for the final image interpretation (e.g., the necessity determination result based on the previous necessity determination rule), and the final image interpretation result from the remote image interpretation DB 40 (S32).

[0084] The control unit 51 applies the acquired image interpretation results and image analysis processing results to the generated necessity determination rule to determine whether the final image interpretation is necessary (S33). Based on the identified necessity determination result and the final image interpretation results based on acquired past medical images, the control unit 51 determines whether the generated necessity determination rule for the final image interpretation needs to be updated (S34).

[0085] The control unit 51 determines whether the impact on the final image interpretation is within an acceptable range (S35). The method for determining whether the impact on the final image interpretation is within an acceptable range will be described later. If the control unit 51 determines that the impact on the final image interpretation is within an acceptable range (YES in S35), it updates the necessity determination rule for the final image interpretation with the generated necessity determination rule (S36) and terminates the process. If the impact on the final image interpretation is not within an acceptable range (NO in S35), the control unit 51 does not update the necessity determination rule for the final image interpretation (S37) and terminates the process. The control unit 51 also functions as a determination unit and may determine the necessity of the final image interpretation using the necessity determination rule generated by the generation unit 57.

[0086] Figure 13 shows an example of the criteria for determining whether the impact on the final interpretation is within an acceptable range. The first criterion is that all medical images with a positive final interpretation result will be judged as requiring final interpretation under the new necessity judgment rules. The update unit 58 can update the necessity judgment rules so that medical images with findings in the final interpretation result before the rule update are judged as requiring final interpretation. This prevents overlooking medical images with findings.

[0087] The second criterion is that the proportion of medical images whose final interpretation result is false positive that are determined not to require final interpretation is equal to or greater than the first threshold (e.g., 15%, 20%). The update unit 58 can update the necessity judgment rule so that the proportion of medical images whose final interpretation result was determined to be "no findings" before the update of the necessity judgment rule is equal to or greater than the first threshold. This makes it possible to suppress the sending of medical images determined to be false positive to final interpretation.

[0088] The third criterion is that the proportion of medical images with a positive final interpretation result that are judged to require further interpretation is equal to or greater than the second threshold (e.g., 95%, 98%). The update unit 58 can update the necessity judgment rule so that the proportion of medical images that were judged to have findings in their final interpretation result before the rule update is judged to require further interpretation is equal to or greater than the second threshold. This increases the likelihood that all medical images judged to be positive will be sent for final interpretation.

[0089] The first to third criteria can be used in whole or in part to determine whether the necessity determination rules are being updated.

[0090] Next, we will explain how to evaluate the updated necessity determination rules once they are actually implemented.

[0091] Figure 14 shows an example of how the server 50 evaluates the necessity determination rule. The control unit 51 acquires multiple medical images to be read and saves them to the remote image reading DB 40 (S41). The control unit 51 acquires the image analysis processing results based on the acquired medical images and saves them to the remote image reading DB 40 (S42). The control unit 51 acquires the image reading results based on the acquired medical images and saves them to the remote image reading DB 40 (S43).

[0092] The control unit 51 obtains the necessity determination result based on the updated final image interpretation necessity determination rule and saves it to the remote image interpretation DB 40 (S44). The control unit 51 obtains the final image interpretation result and saves it to the remote image interpretation DB 40 (S45). The control unit 51 determines whether or not the results necessary for evaluation have been collected (S46), and if they have not been collected (NO in S46), it continues the processing from step S41 onwards.

[0093] If the results necessary for evaluation have been collected (YES in S46), the control unit 51 evaluates the updated final image interpretation necessity determination rule (S47). The control unit 51 determines whether the impact on the final image interpretation is within an acceptable range (S48). The method for determining whether the impact on the final image interpretation is within an acceptable range is illustrated in Figure 13. If the impact on the final image interpretation is within an acceptable range (YES in S35), the control unit 51 decides to continue using the currently operating necessity determination rule and terminates the process. If the impact on the final image interpretation is not within an acceptable range (NO in S48), the control unit 51 reverts the currently operating necessity determination rule to the necessity determination rule before the update (S49) and terminates the process.

[0094] As described above, the control unit 51 has the functions of an evaluation unit and a switching unit. Based on the image interpretation results and detection results from image analysis processing based on medical images, it evaluates the determination results of the necessity determination rules updated by the update unit 58, and can switch between the updated and pre-update necessity determination rules based on the evaluation results. This makes it possible to apply the updated necessity determination rules to past medical images and medical images after operation to determine whether the necessity determination rules currently in operation are appropriate. Furthermore, it becomes possible to change (switch) the necessity determination rules as needed, enabling stable system operation.

[0095] The information processing device of this embodiment includes a first acquisition unit that acquires a determination result of whether or not a final image interpretation is necessary based on a medical image, a second acquisition unit that acquires a final image interpretation result based on a medical image for which a final image interpretation has been determined to be necessary, and a generation unit that generates a necessity determination rule for determining whether or not a final image interpretation is necessary based on the necessity determination result and the final image interpretation result.

[0096] The information processing device of this embodiment includes a third acquisition unit that acquires detection results from image analysis processing based on medical images to be read, and an identification unit that identifies a first group of results in which the final reading result is "finding a finding" and the detection result is "finding a finding," and identifies a second group of results in which the final reading result is "no finding" and the detection result is "finding a finding." The generation unit generates necessity determination rules based on the first and second group results identified by the identification unit.

[0097] In the information processing device of this embodiment, the generation unit generates necessity determination rules such that the proportion of medical images for which the detection result indicates an abnormality is deemed to require final interpretation decreases.

[0098] In the information processing device of this embodiment, the generation unit generates a necessity determination rule using at least one of the detection size, pixel value, and shape indicated by the detection result as a generation condition.

[0099] In the information processing device of this embodiment, the generation unit generates a necessity determination rule using the degree of malignancy of the findings indicated by the detection result as a generation condition.

[0100] The information processing device of this embodiment includes a learning model that outputs a detection target included in a medical image when a medical image is input, and the generation unit generates a necessity determination rule using the confidence level of the detection result output by the learning model as a generation condition.

[0101] The information processing device of this embodiment includes a fourth acquisition unit that acquires image interpretation results based on the medical image, and the generation unit generates necessity determination rules using the number of findings related to the image interpretation results and the detection results as generation conditions.

[0102] In the information processing device of this embodiment, the generation unit generates a necessity determination rule using the type of finding, the location of the finding, or the normal / abnormal determination category indicated by the finding as generation conditions for the image interpretation result and detection result.

[0103] The information processing device of this embodiment includes an update unit that updates the necessity determination rules generated by the generation unit.

[0104] In the information processing device of this embodiment, the update unit updates the necessity determination rule so that, for medical images where the final reading result before the update of the necessity determination rule was "final findings," it is determined that a final reading is necessary.

[0105] In the information processing device of this embodiment, the update unit updates the necessity determination rule so that the percentage of medical images for which the final reading result was determined to be "no findings" before the update of the necessity determination rule is determined to be "no need for final reading" is equal to or greater than a first threshold.

[0106] In the information processing device of this embodiment, the update unit updates the necessity determination rule so that the proportion of medical images for which the final reading result was determined to have findings before the update of the necessity determination rule is determined to require final reading is equal to or greater than the second threshold.

[0107] The information processing device of this embodiment includes an evaluation unit that evaluates the determination result based on the necessity determination rule updated by the update unit, based on the image interpretation result based on the medical image and the detection result from the image analysis processing, and a switching unit that switches between the updated and pre-update necessity determination rules based on the evaluation result from the evaluation unit.

[0108] The information processing device of this embodiment includes a determination unit that determines whether or not a final image interpretation is necessary using the necessity determination rules generated by the generation unit.

[0109] The information processing method of this embodiment obtains a determination result of whether a final image interpretation is necessary based on a medical image, obtains a final image interpretation result based on a medical image for which a final image interpretation has been determined to be necessary, and generates a necessity determination rule for determining whether a final image interpretation is necessary based on the necessity determination result and the final image interpretation result.

[0110] The computer program of this embodiment causes the computer to perform the following processes: obtain the result of determining whether a final image interpretation is necessary based on the medical image; obtain the final image interpretation result based on the medical image for which a final image interpretation was determined to be necessary; and generate a necessity determination rule for determining whether a final image interpretation is necessary based on the necessity determination result and the final image interpretation result.

[0111] The rule generation method of this embodiment acquires detection results from image analysis processing based on medical images to be read, results of determination of whether final reading is necessary, and final reading results. It identifies a first group of data where the final reading result for medical images that are determined to require final reading is positive and the detection result is positive. It identifies a second group of data where the final reading result for medical images that are determined to require final reading is negative and the detection result is positive. Based on the identified first and second group data, it generates rules for determining whether final reading is necessary.

[0112] As described above, this embodiment makes it possible to implement a rule for determining whether or not a final image interpretation is necessary, thereby reducing the burden on physicians by suppressing unnecessary requests for image interpretation from the final interpreting physician.

[0113] In particular, in remote image interpretation services, different interpretation facilities, such as companies and institutions, may use different criteria to determine whether or not an image interpretation is necessary. According to this embodiment, the final interpretation necessity determination rule can be generated according to the required generation conditions, and updates to the necessity determination rule and changes during operation can also be made as appropriate, making it possible to realize an optimal necessity determination rule for each interpretation facility.

[0114] In the above-described embodiment, the server 50 is configured to generate necessity determination rules, update the generated necessity determination rules, and perform the final necessity determination of image interpretation using the generated necessity determination rules. However, these processes may be divided among multiple servers. [Explanation of symbols]

[0115] 1. Communication Network 10. First image reading terminal 20. Second image reading terminal 30 Final image interpretation terminal 40 Remote Image Interpretation Database 50 servers 51 Control Unit 52 Communications Department 53 Interface section 54 Image Processing Unit 55 Recording medium reading unit 551 Recording media 56 memory 57 Generation part 58 Update section 59 Memory section 60 Judging Rules 61 Computer Programs 100, 200, 300 health checkup facilities 101, 102, 103, 150 Terminal devices

Claims

[Claim 1] A first acquisition unit that acquires the result of determining whether or not final image interpretation is necessary based on medical images, A second acquisition unit that acquires the final interpretation result based on medical images that have been determined to require final interpretation, A generation unit generates a necessity determination rule for determining whether the final image interpretation is necessary, based on the necessity determination result and the final image interpretation result. Equipped with, Information processing device.

Images