Diagnosis assistance apparatus, diagnosis assistance program, and diagnosis assistance method

Abstract

To improve accuracy in automatic determination of a subtype carried out on the basis of a pathological image.SOLUTION: A diagnosis assistance apparatus 1 includes: first acquisition means for acquiring a first caption, which describes content of a first pathological image to be diagnosed in a predetermined format, from a first learned model constructed by machine learning so as to generate, in a case where a pathological image is input, a caption, which describes content of the pathological image in the predetermined format; first evaluation means for evaluating a first similarity which is a similarity between content of at least part of a plurality of findings which are accumulated in a database and which represent each of a plurality of pathological subtypes in writing in the predetermined format, and content of the first caption; and output means for outputting information about a subtype whose evaluation of first similarity is highest among the plurality of pathological subtypes. The information output by the output means is used for decision making in diagnosis by a doctor.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a diagnostic support device, a diagnostic support program, and a diagnostic support method. [Background technology]

[0002] Various technologies for assisting doctors in making diagnoses using medical images have been proposed. For example, Patent Document 1 describes an image retrieval device that operates as follows: By receiving input of finding information for a query base image, this image retrieval device derives a query image to which the findings have been added, derives additional finding features for the added findings, derives query normal features that represent image features for normal regions in the query base image, refers to an image database in which multiple reference images are registered, each associated with reference finding features for findings included in the reference image and reference normal features for an image assuming that the findings included in the reference image are normal regions, derives similarities between the query image and multiple reference images based on comparisons of the additional finding features and query normal features with the reference finding features and reference normal features, and extracts reference images similar to the query image from the image database based on the similarities. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2023 / 276432 Summary of the Invention [Problem to be solved by the invention]

[0004] Lesions (for example, skin inflammation and cancer) are classified into countless subtypes, which are classified according to the properties of the cells. When a doctor diagnoses such a lesion, the doctor must determine which subtype the lesion belongs to. However, the conventional technology described in Patent Document 1 does not have sufficient accuracy in deriving features. For this reason, doctors must refer to the extracted similar images, The final decision had to be made by referring to an atlas, as in the past. Determining the subtype using an atlas depends on the doctor's experience and proficiency, which poses problems in terms of accuracy and efficiency.

[0005] The present disclosure has been made in view of the above-mentioned problems, and an exemplary purpose thereof is to improve the accuracy of automatic subtype determination based on pathological images. [Means for solving the problem]

[0006] A diagnostic support device according to an exemplary aspect of the present disclosure includes a first acquisition means for acquiring a first caption C that describes the contents of a first pathological image to be diagnosed in a predetermined format from a first trained model constructed by machine learning to generate a caption C that describes the contents of the pathological image in a predetermined format when the pathological image is input; a first evaluation means for evaluating a first similarity, which is the similarity between the contents of at least a portion of a plurality of findings, each of which is expressed in sentences in the predetermined format for a plurality of pathological subtypes stored in a database, and the contents of the first caption C; and an output means for outputting information regarding the subtype among the plurality of pathological subtypes that has the highest evaluation of the first similarity.

[0007] A diagnostic assistance program according to an exemplary aspect of the present disclosure is a diagnostic assistance program for causing a computer to function as the above-mentioned diagnostic assistance device, and causes the computer to function as the first acquisition means, the first evaluation means, and the output means.

[0008] A diagnostic assistance method according to an exemplary aspect of the present disclosure includes a first acquisition step of acquiring a first caption that describes the contents of a first pathology image to be diagnosed in a predetermined format from a first trained model constructed by machine learning to generate a caption that describes the contents of the pathology image in a predetermined format when the pathology image is input; a first evaluation step of evaluating a first similarity, which is the similarity between the contents of at least a portion of a plurality of findings that are stored in a database and each of a plurality of pathology subtypes are expressed in sentences in the predetermined format, and the contents of the first caption; and an output step of outputting information regarding at least the subtype among the plurality of pathology subtypes that has the highest evaluation of the first similarity. [Effects of the Invention]

[0009] According to an exemplary aspect of the present disclosure, it is possible to improve the accuracy of automatic subtype determination based on pathological images. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 2 is a block diagram illustrating an example of a functional configuration of a diagnosis support device according to a first exemplary embodiment of the present disclosure. [Figure 2] FIG. 10 is a diagram illustrating the function of a first trained model that generates captions acquired by the diagnostic support device. [Figure 3] 10 is a table showing information stored in a database used by the diagnostic support device. [Figure 4] FIG. 2 is a flowchart showing an example of the flow of a diagnosis support method according to a first exemplary embodiment of the present disclosure. [Figure 5] FIG. 10 is a block diagram illustrating an example of a functional configuration of a diagnosis support device according to a second exemplary embodiment of the present disclosure. [Figure 6] FIG. 10 is a diagram illustrating the function of a second trained model that generates feature information acquired by the diagnostic support device. [Figure 7] 10 is a table showing information stored in a database used by the diagnostic support device. [Figure 8] 10 is a table showing an example of correspondence between subtypes stored in a database and first and second evaluation values ​​calculated by the diagnostic support device. [Figure 9] FIG. 10 is a flowchart showing an example of the flow of a diagnosis support method according to a second exemplary embodiment of the present disclosure. [Figure 10] FIG. 11 is a block diagram illustrating an example of a functional configuration of a diagnosis support device according to a third exemplary embodiment of the present disclosure. [Figure 11] FIG. 10 is a flowchart showing an example of the flow of a diagnosis support method according to a third exemplary embodiment of the present disclosure. [Figure 12] FIG. 2 is a block diagram showing the hardware configuration of a computer that functions as a diagnosis support device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, the effects mentioned in the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, embodiments that do not exhibit the effects mentioned in the exemplary embodiments shown below may also be included in the scope of the present invention.

[0012] First Exemplary Embodiment A first exemplary embodiment, which is one example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form of each exemplary embodiment described later. Note that the scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in the drawings referred to in describing this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure to the extent that no particular technical obstacles arise.

[0013] (Configuration of diagnostic support device 1) Next, the configuration of the diagnosis support device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the diagnosis support device 1. As shown in Fig. 1, the diagnosis support device 1 includes a first acquisition means 11, a first evaluation means 12, and an output means 13.

[0014] First Acquisition Method 11 The first acquisition means 11 acquires, from the first trained model M1, a first caption C that explains the content of a first pathological image I to be diagnosed in a predetermined format. The first trained model M1 is constructed by machine learning. Specifically, as shown in FIG. 2 , the first trained model M1 is constructed so that, when a pathological image I is input, it generates a caption C that explains the content of the pathological image I in a predetermined format. The first trained model M1 may be provided in the diagnosis support device 1, or may be provided in another device that communicates with the diagnosis support device 1.

[0015] First evaluation means 12 The first evaluation means 12 evaluates a first similarity, which is the similarity between the content of at least a portion of the multiple findings stored in the database D and the content of the first caption C. The "multiple findings" are sentences expressing each of multiple pathological subtypes in a predetermined format (the same format as the captions). As shown in FIG. 3, the multiple findings are stored in association with the corresponding subtypes. The "at least a portion of the multiple findings" includes "findings related to the same site among the multiple findings," "findings related to the same type of lesion among the multiple findings," "findings excluding clearly unrelated findings among the multiple findings," and the like. The first evaluation means 12 may be configured to express the evaluation result of the first similarity as a numerical value, for example, within a range of 0.0 to 1.0, or may be configured to express it in a manner other than a numerical value (for example, a scale such as strong / medium / weak).

[0016] Output Method 13 The output means 13 outputs information about the subtype with the highest first similarity evaluation among the multiple types of pathological subtypes. The output information about the subtypes is referenced by a doctor making a diagnosis. The doctor who references the information about the subtypes will make a decision about the diagnosis taking the information about the subtypes into consideration.

[0017] (Effects of diagnostic support device 1) The diagnostic support device 1 described above employs a configuration in which the first evaluation means 12 evaluates the first similarity (the similarity between the content of at least a portion of the multiple findings and the content of the first caption C). The diagnostic support device 1 also employs a configuration in which the output means 13 outputs information about the subtype with the highest first similarity evaluation among multiple pathology subtypes. Specifically, the diagnostic support device 1 compares the first caption (corresponding to the finding in the first pathology image) with multiple accumulated findings and presents the finding that most closely resembles the first caption to the user. Because the first trained model M1 is configured to output captions in a predetermined format, the content of the caption for the input pathology image more accurately captures the characteristics of the pathology image. Furthermore, because the findings to be compared are also written in the same predetermined format, the similarity is more accurately evaluated. Therefore, according to the diagnosis support device 1 of this embodiment, it is possible to improve the accuracy of automatic subtype determination based on pathological images (and ultimately reduce the burden on doctors of referring to the atlas).

[0018] (Flow of diagnostic support method S1) Next, the flow of the diagnostic support method S1 will be described with reference to Fig. 4. Fig. 4 is a flow diagram showing the flow of the diagnostic support method S1. As shown in Fig. 4, the diagnostic support method S1 includes a first acquisition step S11, a first evaluation step S12, and an output step S13.

[0019] First acquisition step S11 In a first acquisition step S11, a first caption C that explains the content of a first pathological image I to be diagnosed in a predetermined format is acquired from the first trained model M1. The first caption C can be acquired using, for example, the above-described diagnosis support device 1.

[0020] First evaluation step S12 After the first caption C is acquired, the process proceeds to a first evaluation step S12. In the first evaluation step S12, a first similarity is evaluated. The first similarity is the similarity between the content of at least a portion of the multiple findings stored in the database D and the content of the first caption C. The evaluation of the first similarity may be performed using the above-described diagnosis support device 1 or another device.

[0021] Output step S13 After the first similarity is evaluated, the process proceeds to an output step S13. In the output step S13, information about at least the subtype with the highest evaluation of the first similarity among the multiple types of pathological subtypes is output. The output of information about the subtype with the highest evaluation of the first similarity may be performed using the above-described diagnosis support device 1 or another device.

[0022] (Effects of diagnostic support method S1) As described above, the diagnostic support method S1 employs a configuration in which the first evaluation step S12 evaluates the first similarity (the similarity between the content of at least a portion of the multiple findings and the content of the first caption C). Furthermore, the diagnostic support device 1 employs a configuration in which, among multiple pathology subtypes, information regarding the subtype with the highest evaluation of the first similarity is output in the output step S13. That is, the diagnostic support method S1 compares the first caption (corresponding to the finding in the first pathology image) with multiple accumulated findings, and presents the finding that is closest to the first caption to the user. Because the first trained model M1 is configured to output captions in a predetermined format, the content of the caption for the input pathology image more accurately captures the characteristics of the pathology image. Furthermore, because the findings to be compared are also written in the same predetermined format, the similarity is more accurately evaluated. Therefore, according to the diagnostic support method S1 of this embodiment, it is possible to improve the accuracy of automatic subtype determination based on pathological images (and thereby reduce the burden on doctors of referring to the atlas).

[0023] Second Exemplary Embodiment A second exemplary embodiment, which is one example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be assigned the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in each drawing referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0024] (Configuration of diagnostic support device 1A) Next, the configuration of the diagnosis support device 1A will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the configuration of the diagnosis support device 1A. As shown in Fig. 5, the diagnosis support device 1A according to this embodiment further includes a first evaluation means 12A, a second acquisition means 11A, a second evaluation means 12B, an output means 13A, an input means 14, and a calculation means 15 in addition to the first acquisition means 11 similar to that of the diagnosis support device 1 according to the first exemplary embodiment.

[0025] Input method 14 The input means 14 inputs a prompt for outputting a caption C together with the first pathological image I to the first trained model M1. The input means 14 also inputs the first pathological image I to the second trained model M2. By inputting the prompt to the first trained model M1, the first acquisition means 11 acquires a first caption C in a predetermined format based on the prompt from the first trained model M1. As shown in FIG. 3, the "predetermined format" is a format that specifies the lesion (ccc, calcification, ggg, differentiation, etc.), the location of the lesion (near aaa, between ddd and eee, surrounding aaa, near hhh, etc.), and the condition and degree of the lesion (bbb-like, fff-like, marked, mild, etc.).

[0026] First evaluation means 12A The first evaluation means 12A according to this embodiment calculates a first evaluation value. The first evaluation value is an evaluation result of the first similarity. In other words, the first evaluation value is a numerical representation of the first similarity.

[0027] Second acquisition means 11A The second acquisition means 11A acquires first feature information from the second trained model M2. The first feature information is information indicating the features of the first pathological image I. The second trained model M2 is constructed by machine learning. Specifically, as shown in FIG. 6, the second trained model M2 is constructed so as to generate feature information indicating the features of the pathological image I when the pathological image I is input. The second trained model M2 used by the diagnosis support device 1A according to this embodiment is constructed by object learning or a self-restorer. Therefore, the feature information output by the second trained model is a feature vector V, as shown in FIG. 6. Note that the second trained model M2 may be constructed by object learning or machine learning other than a self-restorer. The feature information may also be in a form other than the feature vector V.

[0028] Second evaluation means 12B The second evaluation means 12B evaluates the second similarity. The second similarity is the similarity between at least a portion of a plurality of pieces of feature information indicating the characteristics of each of a plurality of pathological images I corresponding to each of a plurality of pathological subtypes stored in a database D and the first feature information. As shown in FIG. 7, the plurality of pieces of feature information (feature vectors) are stored in a form linked to the corresponding subtypes and main findings. The database D storing the plurality of pieces of feature information may be the same as or different from the database D described in the first exemplary embodiment. The second evaluation means 12B according to this embodiment calculates a second evaluation value. The second evaluation value is an evaluation result of the second similarity. The second evaluation means 12B according to this embodiment calculates, as the second evaluation value, the cosine similarity between at least a portion of the plurality of feature vectors stored in the database D and the first feature vector, which is the first feature information. Note that the second evaluation means 12B may be configured to calculate a numerical value other than the cosine similarity (e.g., Euclidean distance, Chebyshev distance, etc.) as the second evaluation value.

[0029] The second evaluation value calculated by the second evaluation means 12B and the first evaluation value calculated by the first evaluation means 12A are each linked to a plurality of subtypes stored in the database D, as shown in FIG. 8.

[0030] Calculation method 15 The calculation means 15 calculates a statistical value of the first evaluation value and the second evaluation value. The "statistical value" includes an added value, a weighted added value, an average value, a weighted average value, a selected value, etc. The added value is the sum of the first evaluation value and the second evaluation value. The average value is the average of the first evaluation value and the second evaluation value. The weighted average value is the average of the values ​​obtained by multiplying the first evaluation value and the second evaluation value by a weighting coefficient. The selected value is the larger of the first evaluation value and the second evaluation value.

[0031] Output means 13A The output means 13A according to this embodiment outputs information about the subtype with the highest statistical value among multiple pathological subtypes. "Output" includes displaying the information on a display unit, outputting the information as audio from a speaker, transmitting the information signal to another device from a communication module or the like, and the like. The output means 13A according to this embodiment outputs two or more subtypes among multiple pathological subtypes in descending order of statistical value. Note that the output means 13A may be configured to output at least one of the first evaluation value, the second evaluation value, and the evaluation value together with the subtype.

[0032] (Effects of diagnostic support device 1A) The diagnostic support device 1A described above provides the same effects as the diagnostic support device 1 according to the first exemplary embodiment. Specifically, the diagnostic support device 1A provides the effect of improving the accuracy of automatic subtype determination based on pathological images (and thus reducing the burden of atlas reference work on doctors). The diagnostic support device 1A described above also employs a configuration in which the second acquisition means 11A acquires first feature information from the second trained model M2. The diagnostic support device 1A also employs a configuration in which the second evaluation means 12B evaluates the second similarity (calculates a second evaluation value). The diagnostic support device 1A also employs a configuration in which the output means 13A outputs information on the subtype with the highest statistical value among multiple pathological subtypes. That is, the diagnostic support device 1A outputs the subtype based on two types of evaluation values. Therefore, the diagnostic support device 1A provides the effect of further improving the accuracy of automatic subtype determination based on pathological images.

[0033] (Flow of diagnostic support method S1A) Next, the flow of the diagnostic support method S1A will be described with reference to Fig. 9. Fig. 9 is a flow chart showing the flow of the diagnostic support method S1A. As shown in Fig. 9, the diagnostic support method S1A includes a first acquisition step S11 similar to that of the diagnostic support method S1 according to the first exemplary embodiment, as well as a first evaluation step S12A, a second acquisition step S11A, a second evaluation step S12B, an output step S13A, an input step S14, and a calculation step S15.

[0034] Input step S14 The input step S14 includes a first input step S141 and a second input step S142. The first input step S141 is performed before acquiring the first caption C. In the first input step S141, a prompt for outputting the caption C is input to the first trained model M1 along with the first pathological image I. The second input step is performed before, after, or in parallel with the input of the prompt. In the second input step S142, the first pathological image I is input to the second trained model M2. The first pathological image I and the first caption C can be input using, for example, the above-mentioned diagnosis support device 1A. By inputting the prompt to the first trained model M1, in the first acquisition step S11, a first caption C in a predetermined format based on the prompt is acquired from the first trained model M1.

[0035] First evaluation step S12A After acquiring the first caption C, the process proceeds to a first evaluation step S12A. In the first evaluation step S12A according to this embodiment, a first evaluation value is calculated. The calculation of the first evaluation value may be performed using the above-described diagnosis support device 1A or another device.

[0036] Second acquisition step S11A After calculating the first evaluation value, a second acquisition step S11A is performed before the calculation or in parallel with the calculation. In the second acquisition step S11A, first feature information (feature vector) is acquired from the second trained model M2. The first feature information may be acquired using the above-mentioned diagnosis support device 1A or another device.

[0037] Second evaluation step S12B After the first feature information is acquired, a second evaluation step S12B is performed. In the second evaluation step S12B, a cosine similarity between at least a part of the feature vectors stored in the database D and the first feature vector, which is the first feature information, is calculated as a second evaluation value. The calculation of the second evaluation value may be performed using the diagnosis support device 1A or another device.

[0038] Calculation step S15 After the first evaluation value and the second evaluation value are calculated, the process proceeds to calculation step S15. In calculation step S15, a statistical value of the first evaluation value and the second evaluation value is calculated. The statistical value may be calculated using the diagnosis support device 1A or another device.

[0039] Output step S13A After the statistical values ​​are calculated, an output step S13A is performed. In the output step S13A according to this embodiment, information about the subtype having the highest statistical values ​​of the first evaluation value and the second evaluation value among the multiple types of pathology subtypes is output. In the output step S13A according to this embodiment, two or more subtypes among the multiple types of pathology subtypes are output in descending order of statistical value. The output of the subtypes may be performed using the above-mentioned diagnosis support device 1A or another device.

[0040] (Effects of diagnostic support method S1A) The diagnostic support method S1A described above can achieve the same effect as the diagnostic support method S1 according to the first exemplary embodiment. Specifically, the diagnostic support method S1A can improve the accuracy of automatic subtype determination based on pathological images (and thus reduce the physician's burden of referencing the atlas). The diagnostic support method S1A described above also employs a configuration in which first feature information is acquired from a second trained model M2 in the second acquisition step S11A. The diagnostic support method S1A also employs a configuration in which a second similarity is evaluated (a second evaluation value is calculated) in the second evaluation step S12B. The diagnostic support method S1A also employs a configuration in which information regarding a subtype with the highest statistical value among multiple pathological subtypes is output in the output step S13A. That is, the diagnostic support method S1A outputs a subtype based on two types of evaluation values. Therefore, the diagnostic support method S1A can further improve the accuracy of automatic subtype determination based on pathological images.

[0041] Third Exemplary Embodiment A third exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be assigned the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in each drawing referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0042] (Configuration of diagnosis support device 1B) Next, the configuration of the diagnostic support device 1B will be described with reference to FIG. 10. FIG. 10 is a block diagram showing the configuration of the diagnostic support device 1B. As shown in FIG. 10, the diagnostic support device 1B according to this embodiment includes a first acquisition means 11, an output means 13, and a first evaluation means 12A similar to those of the diagnostic support device 1 according to the first exemplary embodiment, as well as a comparison means 16 and a notification means 17. Note that the diagnostic support device 1B may further include an input means 14, a second acquisition means 11A, a second evaluation means 12B, and a calculation means 15 similar to those of the diagnostic support device 1A according to the second exemplary embodiment.

[0043] ·Comparison means 16 The comparison means 16 compares one or more first evaluation values ​​with a predetermined threshold. Note that, when the diagnosis support device 1B further includes a second acquisition means 11A, a second evaluation means 12B, and a calculation means 15, the comparison means 16 may be configured to compare one or more statistical values ​​with a threshold.

[0044] ·Notification method 17 When the largest first evaluation value among the first evaluation values ​​is less than the threshold, the notification means 17 notifies information to that effect. The "information to the effect (that the largest statistical value is less than the threshold)" includes information such as the statistical value being less than the threshold (status), the inability to perform a normal evaluation (error), the inappropriateness of at least one of the first pathological image I and the prompt (cause), and the need to change at least one of the first pathological image I and the prompt and try again (solution).

[0045] (Effects of diagnostic support device 1B) The diagnostic support device 1B described above provides the same effect as the diagnostic support device 1 according to the first exemplary embodiment. Specifically, the diagnostic support device 1B provides the effect of improving the accuracy of automatic subtype determination based on pathological images (and thus reducing the burden of atlas reference work on doctors). The diagnostic support device 1B described above also employs a configuration in which the comparison means 16 compares one or more first evaluation values ​​with a predetermined threshold. The diagnostic support device 1B also employs a configuration in which, if the largest first evaluation value is less than the threshold, the notification means 17 notifies the user of information indicating that the largest first evaluation value is less than the threshold. Therefore, the diagnostic support device 1B provides the effect of making the user aware of any defects in at least one of the first pathological image and the prompt input to the first trained model M1.

[0046] (Flow of diagnostic support method S1B) Next, the flow of the diagnostic support method S1B will be described with reference to FIG. 11. FIG. 11 is a flow chart showing the flow of the diagnostic support method S1B. As shown in FIG. 11, the diagnostic support method S1B according to this embodiment includes a first acquisition step S11, an output step S13, and a first evaluation step S12A similar to the diagnostic support method S1A according to the first exemplary embodiment, as well as a comparison step S16 and a notification step S17. Note that the diagnostic support method S1B may further include an input step S14, a second acquisition step S11A, a second evaluation step S12B, and a calculation step S15 similar to the diagnostic support method S1A according to the second exemplary embodiment.

[0047] Comparison step S16 After the first evaluation values ​​are calculated, a comparison step S16 is performed. In the comparison step S16, one or more first evaluation values ​​are compared with a predetermined threshold. The comparison of the first evaluation values ​​with the threshold may be performed using the above-mentioned diagnosis support device 1A or another device. If at least the largest first evaluation value among the first evaluation values ​​is equal to or greater than the threshold (S16: NO), the process proceeds to an output step. Note that if the diagnosis support method S1B further includes a second acquisition step S11A, a second evaluation step S12B, and a calculation step S15, the comparison step S16 may compare one or more statistical values ​​with the threshold.

[0048] Notification step S17 If the largest first evaluation value among the first evaluation values ​​is less than the threshold value (S16: YES), a notification step S17 is performed. In the notification step S17, information indicating that the largest first evaluation value is less than the threshold value is notified. The notification may be performed using the diagnosis support device 1A or another device. After notifying the information, the process returns to the input step S14. In the second or subsequent input step S14, at least one of the first pathological image I and the prompt is changed to one different from the previous time and input into the first trained model M1.

[0049] (Effect of diagnostic support method S1B) The diagnostic support method S1B described above can achieve the same effect as the diagnostic support method S1 according to the first exemplary embodiment. Specifically, the diagnostic support method S1B can improve the accuracy of automatic subtype determination based on pathological images (and thus reduce the burden of atlas reference work on doctors). The diagnostic support method S1B described above also employs a configuration in which one or more first evaluation values ​​are compared with a predetermined threshold in the comparison step S16. The diagnostic support method S1B also employs a configuration in which, if the largest first evaluation value is less than the threshold in the notification step S17, information to that effect is notified. Therefore, the diagnostic support method S1B can notify the user of any defects in at least one of the first pathological image and the prompt input to the first trained model M1.

[0050] [Software implementation example] Some or all of the functions of the diagnosis support devices 1, 1A, 1B (hereinafter also referred to as "each of the above devices") may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0051] In the latter case, each of the above devices is realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 12. Figure 12 is a block diagram showing the hardware configuration of computer C that functions as each of the above devices.

[0052] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a diagnostic assistance program P for causing the computer C to function as each of the above-mentioned means. In the computer C, the processor C1 reads and executes the diagnostic assistance program P from the memory C2, thereby realizing the functions of each of the above-mentioned means.

[0053] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0054] The computer C may further include a RAM (Random Access Memory) for expanding the diagnostic assistance program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0055] The diagnostic assistance program P can also be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the diagnostic assistance program P via such a recording medium M. The diagnostic assistance program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the diagnostic assistance program P via such a transmission medium.

[0056] [Appendix 1] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0057] (Appendix 1) a first acquisition means for acquiring, from a first trained model constructed by machine learning so as to generate a caption that explains the content of a first pathology image in a predetermined format when the pathology image is input, a first caption that explains the content of the first pathology image in the predetermined format; a first evaluation means for evaluating a first similarity, which is the similarity between the content of at least a part of a plurality of findings, each of which expresses a plurality of pathological subtypes in a sentence in a predetermined format and which are stored in a database, and the content of the first caption; an output means for outputting information about the subtype having the highest evaluation of the first similarity among the plurality of types of pathological subtypes; Equipped with Diagnostic support device.

[0058] (Appendix 2) a second acquisition means for acquiring first feature information indicating features of the first pathological image from a second trained model constructed by machine learning so as to generate feature information indicating features of the pathological image when the pathological image is input; a second evaluation means for evaluating a second similarity, which is a similarity between at least a portion of a plurality of pieces of feature information indicating features of a plurality of pathological images corresponding to each of a plurality of types of pathology subtypes stored in a database, and the first feature information; Furthermore, the output means outputs information about a subtype having the highest statistical value of a first evaluation value, which is the evaluation result of the first similarity, and a second evaluation value, which is the evaluation result of the second similarity, among the plurality of types of pathology subtypes. 2. The diagnostic support device according to claim 1.

[0059] (Appendix 3) the output means outputs two or more subtypes from among the plurality of types of pathological subtypes in descending order of the statistical value; 3. The diagnostic support device according to claim 1 or 2.

[0060] (Appendix 4) an input means for inputting a prompt regarding output of the caption together with the first pathological image to the first trained model; the first acquisition means acquires the first caption based on the prompt from the first trained model; 4. A diagnostic support device according to any one of appendices 1 to 3.

[0061] (Appendix 5) a comparison means for comparing one or more of the first evaluation values ​​with a predetermined threshold; a notification means for notifying information indicating that the largest first evaluation value among the first evaluation values ​​is less than the threshold value; Further provided with 3. The diagnostic support device according to claim 2.

[0062] (Appendix 6) the feature information is a feature vector, the second evaluation means calculates, as the second evaluation value, a cosine similarity between at least a part of a plurality of feature vectors stored in a database and a first feature vector that is the first feature information; 3. The diagnostic support device according to claim 2.

[0063] (Appendix 7) A diagnostic support program for causing a computer to function as the diagnostic support device according to any one of appendices 1 to 6, A diagnostic support program for causing a computer to function as the first acquisition means, the first evaluation means, and the output means.

[0064] (Appendix 8) a first acquisition step of acquiring a first caption that describes the content of a first pathology image to be diagnosed in a predetermined format from a first trained model that is constructed by machine learning so as to generate a caption that describes the content of the pathology image in a predetermined format when the pathology image is input; a first evaluation step of evaluating a first similarity, which is the similarity between the content of at least a part of a plurality of findings, each of which expresses a plurality of types of pathology subtypes in the predetermined format and which are stored in a database, and the content of the first caption; an output step of outputting information about at least the subtype having the highest evaluation of the first similarity among the plurality of types of pathology subtypes; Including, Diagnostic support methods.

[0065] (Appendix 9) a second acquisition step of acquiring first feature information indicating features of the first pathological image from a second trained model constructed by machine learning so as to generate feature information indicating features of the pathological image when the pathological image is input; a second evaluation step of evaluating a second similarity, which is a similarity between at least a portion of a plurality of pieces of feature information indicating features of each of a plurality of pathological images corresponding to each of a plurality of types of pathology subtypes stored in a database, and the first feature information; further comprising In the output step, information regarding the subtype having the highest statistical value of a first evaluation value that is the evaluation result of the first similarity and a second evaluation value that is the evaluation result of the second similarity is output from among the multiple types of pathological subtypes. A diagnostic assistance method as described in Appendix 8.

[0066] (Appendix 10) In the output step, two or more subtypes among the plurality of types of pathological subtypes are output in descending order of the statistical value. A diagnostic support method according to appendix 8 or 9.

[0067] (Appendix 11) The method further includes an input step of inputting a prompt regarding output of the caption together with the first pathological image to the first trained model; In the first obtaining step, the first caption based on the prompt is obtained from the first trained model. 11. A diagnostic support device according to any one of appendices 8 to 10.

[0068] (Appendix 12) a comparing step of comparing one or more of the first evaluation values ​​with a predetermined threshold; a notification step of notifying information indicating that the largest first evaluation value among the first evaluation values ​​is less than the threshold value; further comprising: 10. A diagnostic assistance method according to claim 9.

[0069] (Appendix 13) the feature information is a feature vector, In the second evaluation step, a cosine similarity between at least a part of a plurality of feature vectors stored in a database and a first feature vector that is the first feature information is calculated as the second evaluation value. 10. A diagnostic assistance method according to claim 9.

[0070] (Appendix 14) A diagnostic support program for causing a computer to function as the diagnostic support device according to any one of appendices 1 to 6, A diagnostic support program for causing a computer to function as the first acquisition means, the first evaluation means, and the output means.

[0071] [Appendix 2] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0072] (Appendix 1) at least one processor, a first acquisition process for acquiring a first caption that describes the content of a first pathology image to be diagnosed in a predetermined format from a first trained model that is constructed by machine learning so as to generate a caption that describes the content of the pathology image in a predetermined format when the pathology image is input; a first evaluation process for evaluating a first similarity, which is the similarity between the content of at least a part of a plurality of findings, each of which represents a plurality of pathological subtypes in a sentence in a predetermined format and which are stored in a database, and the content of the first caption; an output process of outputting information about the subtype having the highest evaluation of the first similarity among the plurality of types of pathological subtypes; To execute Diagnostic support device.

[0073] (Appendix 2) The at least one processor: a second acquisition process for acquiring first feature information indicating features of the first pathological image from a second trained model constructed by machine learning so as to generate feature information indicating features of the pathological image when the pathological image is input; a second evaluation process for evaluating a second similarity, which is a similarity between at least a portion of a plurality of pieces of feature information indicating features of a plurality of pathological images corresponding to each of a plurality of types of pathology subtypes stored in a database, and the first feature information; Further execute In the output process, the at least one processor outputs information about a subtype having the highest statistical value of a first evaluation value, which is the evaluation result of the first similarity, and a second evaluation value, which is the evaluation result of the second similarity, among a plurality of types of pathology subtypes. 2. The diagnostic support device according to claim 1.

[0074] (Appendix 3) In the output process, the at least one processor outputs two or more subtypes from among the plurality of types of pathology subtypes in descending order of the statistical value. 3. The diagnostic support device according to claim 1 or 2.

[0075] (Appendix 4) The at least one processor: further performing an input process of inputting a prompt regarding output of the caption together with the first pathological image to the first trained model; In the first acquisition process, the at least one processor acquires the first caption based on the prompt from the first trained model. 4. A diagnostic support device according to any one of appendices 1 to 3.

[0076] (Appendix 5) The at least one processor: a comparison process for comparing one or more of the first evaluation values ​​with a predetermined threshold; a notification process for notifying information indicating that the maximum first evaluation value among the first evaluation values ​​is less than the threshold value; Further implementation of 3. The diagnostic support device according to claim 2.

[0077] (Appendix 6) the feature information is a feature vector, In the second evaluation process, the at least one processor calculates, as the second evaluation value, a cosine similarity between at least a portion of a plurality of feature vectors stored in a database and a first feature vector that is the first feature information. 3. The diagnostic support device according to claim 2. [Explanation of symbols]

[0078] 1, 1, 1A, 1B, 1A, 1B Diagnostic support device 11 First acquisition method 11A Secondary Acquisition Method 12, 12A First evaluation means 12B Secondary Assessment Tool 13, 13A output means 14 Input Methods 15 Calculation method 16 Means of comparison 17 Means of Notification C1 processor C2 Memory C Computer C1 processor C2 Memory D Database M Recording medium M1 First trained model M2 Second trained model P Diagnostic Support Program S1, S1A, S1B diagnostic support methods S11 First acquisition step S11A Second acquisition step S12, S12A First evaluation step S12B Second Evaluation Step S13, S13A output step S14 Input step S141 First input step S142 Second input step S15 Calculation step S16 Comparison step S17 Notification Step

Claims

1. a first acquisition means for acquiring, from a first trained model constructed by machine learning so as to generate a caption that explains the content of a pathology image in a predetermined format when the pathology image is input, a first caption that explains the content of the first pathology image in the predetermined format; a first evaluation means for evaluating a first similarity, which is the similarity between the content of at least a part of a plurality of findings, each of which represents a plurality of pathological subtypes in a sentence in a predetermined format and which are stored in a database, and the content of the first caption; an output means for outputting information about the subtype having the highest evaluation of the first similarity among the plurality of types of pathological subtypes; Equipped with Diagnostic support device.

2. a second acquisition means for acquiring first feature information indicating features of the first pathological image from a second trained model constructed by machine learning so as to generate feature information indicating features of the pathological image when the pathological image is input; a second evaluation means for evaluating a second similarity, which is a similarity between at least a portion of a plurality of pieces of feature information indicating features of a plurality of pathological images corresponding to each of a plurality of types of pathology subtypes stored in a database, and the first feature information; Furthermore, the output means outputs information about a subtype having the highest statistical value of a first evaluation value, which is the evaluation result of the first similarity, and a second evaluation value, which is the evaluation result of the second similarity, among the plurality of types of pathology subtypes. The diagnosis support device according to claim 1 .

3. the output means outputs two or more subtypes from among the plurality of types of pathological subtypes in descending order of the statistical value; The diagnosis support device according to claim 2 .

4. an input means for inputting a prompt regarding output of the caption together with the first pathological image to the first trained model; the first acquisition means acquires the first caption based on the prompt from the first trained model; The diagnosis support device according to claim 1 .

5. a comparison means for comparing one or more of the first evaluation values ​​with a predetermined threshold; a notification means for notifying information indicating that the maximum first evaluation value among the first evaluation values ​​is less than the threshold value; Further provided with The diagnosis support device according to claim 2 .

6. the feature information is a feature vector, the second evaluation means calculates, as the second evaluation value, a cosine similarity between at least a part of a plurality of feature vectors stored in a database and a first feature vector that is the first feature information; The diagnosis support device according to claim 2 .

7. A diagnostic support program for causing a computer to function as the diagnostic support device according to claim 1, A diagnostic support program for causing a computer to function as the first acquisition means, the first evaluation means, and the output means.

8. a first acquisition step of acquiring a first caption that describes the content of a first pathology image to be diagnosed in a predetermined format from a first trained model that is constructed by machine learning so as to generate a caption that describes the content of the pathology image in a predetermined format when the pathology image is input; a first evaluation step of evaluating a first similarity, which is the similarity between the content of at least a part of a plurality of findings, each of which expresses a plurality of types of pathology subtypes in the predetermined format and which are stored in a database, and the content of the first caption; an output step of outputting information about at least the subtype having the highest evaluation of the first similarity among the plurality of types of pathology subtypes; Including, Diagnostic support methods.

Images