Image generation device, image generation method, display device, image generation program, and recording medium

Abstract

And the credibility of the analysis result obtained by using the machine model is further improved. An image generation device is provided with: an acquisition unit that acquires an analysis result output from a machine model that analyzes a medical image obtained by capturing an image of a subject, and derivation basis data indicating a basis for deriving the analysis result; and an image generation unit that generates a display image in which the medical image has been changed on the basis of the derivation basis data.

Description

Technical Field

[0001] The present disclosure relates to an image generation device, an image generation method, a display device, an image generation program, and a recording medium. Background Art

[0002] For example, Patent Document 1 discloses an X-ray age estimation model that uses data of an X-ray image of a subject and information on the age of the subject when the X-ray image was taken, to show the correspondence between the feature amount obtained from the data of the X-ray image and the age of the subject.

[0003] Prior Art Documents

[0004] Patent Documents

[0005] Patent Document 1: Japanese Patent Laid-Open Publication No. 2022-148729 Summary of the Invention

[0006] -Problems to be Solved by the Invention-

[0007] In recent years, various images have been analyzed using a learned machine model. However, the basis for most of the analysis results obtained using the machine model is unknown. An object of the present disclosure is to further improve the credibility of the analysis results obtained using the machine model.

[0008] -Means for Solving the Problems-

[0009] An image generation device according to one aspect of the present disclosure includes: an acquisition unit that acquires an analysis result output from a machine model that analyzes a medical image of a subject and derivation basis data indicating the basis for deriving the analysis result; and an image generation unit that generates a display image obtained by changing the medical image based on the derivation basis data.

[0010] An image generation method according to one aspect of the present disclosure includes the steps of: an acquisition step of acquiring an analysis result output from a machine model that analyzes a medical image of a subject and derivation basis data indicating the basis for deriving the analysis result; and an image generation step of generating a display image obtained by changing the medical image based on the derivation basis data.

[0011] The image generation device according to each aspect of the present disclosure can also be implemented by a computer. In this case, a control program for causing a computer to function as each unit (software element) included in the image generation device and a computer-readable recording medium recording the control program are also included in the scope of the present disclosure.

[0012] -Effects of the Invention-

[0013] According to one aspect of the present disclosure, it is possible to further improve the reliability of the analysis result obtained by using a machine model. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a block diagram showing the configuration of an image generation device according to Embodiment 1 of the present disclosure.

[0015] Figure 2 It is a flowchart showing the process of an image display method according to Embodiment 1.

[0016] Figure 3 It is a schematic diagram showing an example of a display image according to Embodiment 1.

[0017] Figure 4 It is a block diagram showing the configuration of an image remote analysis system according to Embodiment 2 of the present disclosure.

[0018] Figure 5 It is a schematic diagram showing an example of a browser image for user input.

[0019] Figure 6 It is a schematic diagram showing an example of X-ray image data for commissioning bone density analysis.

[0020] Figure 7 It is a schematic diagram showing an example of a browser image of the analysis result sent by the analysis result sending unit.

[0021] Figure 8 It is shown in Figure 7 A schematic diagram showing an example of a browser image in which derivation basis data is added to the display image shown.

[0022] Figure 9 It is Figure 8 An enlarged schematic diagram of the image of the area shown.

[0023] Figure 10 It is a schematic diagram showing an example of a browser image displaying the current time and the future possibility of a patient's fracture.

[0024] Figure 11 It is shown in Figure 10 A schematic diagram showing an example of a browser image in which derivation basis data is added to the display image shown.

[0025] Figure 12 It is a schematic diagram showing an example of a display image to which a cytopathological analysis result and its derivation basis data are added.

[0026] Figure 13 It is a schematic diagram showing an example of scrolling to display a display image on one screen.

[0027] Figure 14 It is a block diagram showing the structure of the image remote analysis system related to Embodiment 3.

[0028] Figure 15 It is an example of an image that is the basis data for the derivation of the change of an image including changes based on other reasons analyzed by the machine model.

[0029] Figure 16 It is an example of a browser image showing the analysis results in two cases, including the case of the change of the image based on other reasons and the case of excluding the change of the image. Detailed Embodiment

[0030] 〔Embodiment 1〕

[0031] Hereinafter, an embodiment of the present disclosure will be described in detail with reference to the accompanying drawings. Figure 1 It is a block diagram showing the structure of the image generation device 3 related to Embodiment 1 of the present disclosure. The image generation device 3 is a device that generates a display image of a medical image changed in a manner including the basis of the analysis result based on the region of interest information related to the region of interest that is concerned in the process of outputting the analysis result from the machine model for analyzing the medical image. The image generation device 3 can perform information communication with the analysis device 60 and the display device 70 via the communication unit 50. The analysis device 60 includes a machine model 601. The machine model 601 is, for example, a trained machine model that analyzes at least one of an image and a numerical value. The display device 70 displays, for example, the result analyzed by the machine model 601.

[0032] As Figure 1 shown, the image generation device 3 includes a control unit 30, a storage unit 40, and a communication unit 50. The control unit 30 includes an image generation unit 31, a data acquisition unit (acquisition unit) 32, and a communication control unit 33.

[0033] The data acquisition unit 32 acquires the analysis result (analysis data) output (derived) from the machine model 601 of the analysis image and the data on the basis for deriving the analysis result. As an example, the data on the basis for deriving is the region-of-interest information related to the region within the medical image, that is, the region of interest that is concerned in the process of outputting the analysis result. The data acquisition unit 32 records the acquired analysis data 41 and the data on the basis for deriving 42 in the storage unit 40. The medical image includes, for example, at least one of a plain X-ray image, a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, a PET (Positron Emission Tomography) image, and an ultrasonic image. When using a plain X-ray image as the medical image, the imaging part of the plain X-ray image is not particularly limited. In the plain X-ray image used as the medical image, for example, an image showing a part of the bones of the subject can be used. More specifically, the plain X-ray image only needs to photograph at least any one of the head, neck, chest, waist, hip joint, knee joint, ankle joint, foot, toes, shoulder joint, elbow joint, hand joint, hand, finger, or jaw joint of the subject. In addition, the plain X-ray image for estimation is used to estimate the bone density and fracture site of the bones of the subject. Such a plain X-ray image for estimation can be a front view showing the subject part from the front (for example, an image obtained by irradiating the subject part with X-rays in the anteroposterior direction), or a side view showing the subject part from the side (for example, an image obtained by irradiating the subject part with X-rays in the left-right direction). When using a CT image as the medical image, for example, at least one of a three-dimensional image, a cross-sectional image in a direction perpendicular to the body axis connecting the head and the legs (for example, a horizontal cross-section), and a cross-sectional image in a direction parallel to the body axis (for example, a sagittal plane or a coronal plane, etc.) can be used. The medical image can be an image showing bones. The medical image can be an image not showing bones. The bone density of the bones can be represented by at least one of the bone density per unit area (g / cm 2 ), the bone density per unit volume (g / cm 3 ), YAM (%), T-score, and Z-score. YAM (%) is the abbreviation of "Young Adult Mean" and is sometimes called the young adult average percentage. For example, the bone density of the bones can be a value represented by the bone density per unit area (g / cm 2 ) and YAM (%). The bone density of the bones can be either an index determined by a guideline or an original index.

[0034] The image generation unit 31 generates a display image (display image data) of the medical image changed based on the region-of-interest information. The image generation unit 31 records the generated display image data 43 in the storage unit 40.

[0035]

[0035] The communication control unit 33 sends the display image data 43 recorded in the storage unit 40 to the display device 70. In this case, the communication control unit 33 may also send the display image data 43 to the display device 70. For example, the communication control unit 33 may first send the image data used in the analysis and then send the analysis data 41. Alternatively, for example, the communication control unit 33 may first send the analysis data 41 and then send the image data used in the analysis.

[0036]

[0036] The control unit 30 uniformly controls the entirety of the image generation device 3. The control unit 30 includes at least one processor and at least one memory. The processor can be constituted by, for example, at least one general-purpose processor such as an MPU (Micro Processing Unit) or a CPU (Central Processing Unit). The memory may also include various memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory). As an example, the processor realizes the functions of the image generation device 3 by expanding and executing various control programs recorded in the ROM of the memory in the RAM. In addition, the processor may also include a processor constituted by an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a PLD (Programmable Logic Device), etc.

[0037] Figure 2 Figure 2 This is a flowchart showing the flow of the image display method S1 using the image generation device 3 as an example. As shown in the figure, the image display method S1 includes steps S11 to S13.

[0038] The data acquisition unit 32 acquires the analysis result output from the machine model 601 that analyzes the medical image obtained from the subject, and the derivation basis data indicating the basis for deriving the analysis result (S11). Specifically, for example, the data acquisition unit 32 acquires the analysis result output from the machine model that analyzes the medical image obtained from the subject, and the region-of-interest information related to the region within the medical image and the region of interest that is concerned during the output of the analysis result. The region within the medical image may be, for example, a region closer to the inside than the outer periphery of the medical image. The region-of-interest information may, for example, all be located within the region of the medical image or only partially overlap. Alternatively, when the region-of-interest information does not exist in the medical image (or, for example, when the region-of-interest information is the entire medical image), the region-of-interest information may not overlap with the region in the medical image. In this case, for example, the meaning of the non-existence of the region of interest (or, for example, the meaning that the entire medical image is the region of interest) may be displayed at a position outside the region within the medical image. In the following description, the case where the subject is a person (i.e., "subject") is taken as an example, but the subject is not limited to a person. The subject may be, for example, a mammal other than a person such as a horse, a cat, a dog, a cow, or a pig. And as long as the present disclosure can be applied to the embodiments of these animals in the following embodiments, it also includes embodiments in which "subject" is changed to "animal".

[0039] The image generation unit 31 generates a display image of the medical image changed based on the acquired derivation basis data (S12). Specifically, the image generation unit 31 generates, for example, a display image of the medical image changed based on the region-of-interest information.

[0040] The communication control unit 33 sends the display image generated by the image generation unit 31 to the display device 70 (step S13). Through the above processing method, the analysis result derived from the machine model that analyzes the image and the derivation basis data of the analysis result can be displayed on one screen. Through this method, as an example, generate Figure 3 a display image 100 as shown, which displays the analysis result output from the machine model that analyzes the medical image obtained from the subject and the region-of-interest information related to the region within the medical image and the region of interest that is concerned during the output of the analysis result on one screen, and is displayed on the display device 70.

[0041] Figure 3 It is a schematic diagram showing an example of the display image (display screen) 100 according to Embodiment 1. Specifically, Figure 3It is a display image that shows the analysis results derived from a machine model that analyzes an image and the data on which the analysis results are based. The data on which the analysis results are based refers to at least one of the reasons why the machine model derived such analysis results. For example, the data on which the analysis results are based can be information that mainly indicates which part of the image (medical image) the analysis results are derived from, or information that indicates what degree of weight is set in which part to derive the results. The display image refers to an image displayed on the screen (display screen) of a display device. The display image can also be a browser image, that is, an image displayed on a website opened on the Internet that can be viewed using a browser. The type of the display device is not limited. The screen can be, for example, the screen of a fixed personal computer, the screen of a mobile terminal, etc.

[0042] In Figure 3 specifically, the analysis results derived from the machine model 601 that analyzes an image including bones and the data on which the analysis results are based are displayed on a display screen (the screen that displays the display image 100). The analysis results are displayed in the area 101 and include bone density and the ratio when comparing the bone density with the average bone density of young adults, etc. The data on which the analysis results are based is displayed in the area 102. What is displayed in the area 102 is an image with the area of interest 103 added to the image including bones analyzed by the machine model 601. In this embodiment, the area of interest is the area that is the main basis for deriving the analysis results of the image. The "Predict the Future" button 105, "Return" button 106, and "End" button 107 will be described later.

[0043] As Figure 3 shown, it is possible to display the analysis results derived from the machine model 601 that analyzes the image and the data on which the analysis results are based on one screen. Thus, the user can confirm, together with the analysis results, at least one of the reasons why the machine model derived such analysis results.

[0044] By using the above image generation device 3 or image display method S1, it is possible to display the analysis results derived from the machine model that analyzes the image and the data on which the analysis results are based on one screen. Thus, the user can confirm, together with the analysis results, one of the reasons why the machine model derived such analysis results. Therefore, compared with the case of only displaying the analysis results, the credibility of the analysis results obtained using the machine model can be further improved.

[0045] 〔Embodiment 2〕

[0046] Other embodiments of the present disclosure will be described below. In this embodiment, an analysis system (also referred to as an image remote analysis system) 70A that has received an analysis commission of an image from a user analyzes an X-ray image of bones and outputs, for example, bone density, relative comparison of bone density, possibility of fracture (%), etc., and an example of a display image will be described. The image remote analysis system 70A can display to the user, together with the analysis result, one of the bases for why the machine model has derived such an analysis result.

[0047] (Structure of the image remote analysis system 70A)

[0048] Figure 4 It is a block diagram showing the structure of the image remote analysis system 70A according to Embodiment 2 of the present disclosure. As shown in the figure, the image remote analysis system 70A includes an analysis device 60A (image remote analysis device) that analyzes medical data (for example, medical image data) and a user terminal 20, and the analysis device 60A and the user terminal 20 are communicably connected via the Internet.

[0049] The analysis device 60A includes a communication unit 15, a control unit 16, and a storage unit 17. The communication unit 15 communicates with the user terminal 20 via the Internet. The storage unit 17 stores acquisition information 172 including image data 301, encrypted patient information 302, and attribute information 303, and analysis result data 173 including processed image data 310 and derived basis data 311. The storage unit 17 associates and stores the image data 301, encrypted patient information 302, and attribute information 303 acquired by the acquisition unit 12.

[0050] The control unit 16 includes a communication control unit 11 (analysis result sending unit), an acquisition unit 12, an analysis unit 13, a generation unit (image generation unit) 18, and a determination unit 19. The communication control unit 11 controls the communication unit 15. The acquisition unit 12 acquires the data transmitted from the user terminal 20 via the communication control unit 11. In addition, the acquisition unit 12 acquires the analysis result (analysis data) derived by the analysis unit 13 (machine model 131) that analyzes the medical image and the derived basis data of the analysis result. The analysis unit 13 analyzes the image data 301. The control unit 16 may have the same structure as the control unit 30 described in Embodiment 1.

[0051] The generation unit 18 generates derivation basis data for the analysis result analyzed by the analysis unit 13. The derivation basis data 311 refers to at least one of the bases for why the machine model derived such an analysis result. For example, the derivation basis data 311 can also be information mainly indicating which part of the image the analysis result was derived from. The derivation basis data 311 is at least associated with the encrypted patient information 302 and stored in the storage unit 17. A specific example of the function of the generation unit 18 will be described later. In addition, the generation unit 18 generates a display image (display image data) that displays the analysis result obtained by the acquisition unit 12 and the region of interest information (derivation basis data for the analysis result) on one screen. The generation unit 18 records the generated display image data in the storage unit 17.

[0052] (Determination unit 19)

[0053] The determination unit 19 determines whether the encrypted patient information 302 obtained by the acquisition unit 12 is information obtained by actually encrypting the patient identification information 320. In the case where it is determined that the encrypted patient information 302 is not information that has been actually encrypted, the acquisition unit 12 may also delete the obtained encrypted patient information 302. By providing such a determination unit 19, assuming that the encrypted patient information 302 sent from the user terminal 20 is not information obtained by encrypting the patient identification information 320, the acquired image data 301 is not analyzed and is deleted together with the encrypted patient information 302 and the attribute information 303. Therefore, the acquisition information 172 including the image data 301 is not stored inside the analysis device 60A. The determination of whether it is encrypted information can be made, for example, based on the file extension of the encrypted data.

[0054] In the case where it is determined that the encrypted patient information 302 is not information that has been actually encrypted, the control unit 16 may also send a message to the user terminal 20 via the communication control unit 11 to request re - encryption and transmission of the patient identification information 320.

[0055] (Browser image 200)

[0056] Figure 5 It is a schematic diagram showing an example of the browser image 200 that is displayed on a web page for inputting the image data 301 etc. when the user accesses the analysis device 60A via the Internet.

[0057] As shown in the figure, in the browser image 200, text such as <Bone Density Analysis>, which represents the object of image analysis, is displayed at the top. The text displayed at the top is not limited to this. For example, it can also be a display of known analysis content, such as <Bone Density Estimation> and <Predict Future Bone Density>, etc. In the upper left part of the browser image 200, it is also possible to record the stage of the current screen being analyzed or status information of known analysis content. For example, in the upper left part of the browser image 200, <Received>, <Analysis Result>, <Predict Future>, <Benign / Malignant Judgment>, etc. can be used, but it is not limited to this.

[0058] In addition, in the upper left part of the browser image 200, for example, text such as <Received>, which indicates that it is a screen for receiving input, is displayed. Below it, text such as "Place the image to be analyzed here" that urges the input of image data to be analyzed and a frame 205 representing the area for pasting (dragging and dropping) image data are displayed. And below it, text such as "Please enter attribute information here" that urges the input of patient identification information 320, attribute information 303, etc. and a check box 206 for input are displayed. A "Send" button 204 that urges sending is displayed in the lower right part of the browser image 200, and a "Return" button 207 that returns to the initial screen of the website is displayed in the upper right part.

[0059] Figure 6FIG. 0 is a schematic diagram showing X-ray image data 300 for commissioning bone density analysis as an example of image data 301. The X-ray image data 300 may be a simple X-ray image such as a lumbar X-ray image or a chest X-ray image, or may also be an X-ray image taken by a DXA (Dual energy X-ray Absorptiometry) device or the like. In a DXA device that measures bone density using the DXA method, when measuring the bone density of the lumbar spine, X-rays are irradiated onto the lumbar spine of the subject from the front. In addition, in a DXA device, when measuring the bone density of the proximal femur, X-rays are irradiated onto the proximal femur of the subject from the front. The ultrasonic method is a method of measuring bone density by applying ultrasonic waves to bones such as the heel and the calf. Here, "the front with respect to the lumbar spine" and "the front with respect to the proximal femur" refer to the directions that correctly face the imaging parts such as the lumbar spine and the proximal femur, and may be the ventral side of the subject's body or the dorsal side of the subject. The MD (micro densitometry) method irradiates X-rays onto the hand. The ultrasonic method is a method of measuring bone density by applying ultrasonic waves to bones such as the lumbar spine, the femur, the heel, or the calf. In addition, even if the image data is not an X-ray image, as long as it is an image including bone information. The image data can be estimated from, for example, MRI (magnetic resonance imaging) images, CT (computed tomography) images, PET images, and ultrasonic images.

[0060] In the present embodiment, when the user accesses the analysis device 60A via the Internet, the communication control unit 11 may also display the browser image 200 on the web page and send the encryption application via the communication unit 15. When the user inputs the image data 301 for commissioning analysis, the patient identification information 320, and the attribute information 303 and clicks the "Send" button 204, the patient identification information 320 is encrypted, and the encrypted encrypted patient information 302 is sent to the analysis device 60A together with the image data 301 and the attribute information 303. Therefore, the user does not need to perform the encryption process of the patient identification information 320. The encryption process may also be executed not when the user clicks the "Send" button 204, but after the click.

[0061] The encryption method may be a known method and is not limited. The encryption method may also be, for example, an encryption method that combines a public key and a secret key. The encryption may also be a method that cannot be decrypted even by the operator of the image remote analysis system 70A. Thus, even when the user commissions image analysis to the operator of the image remote analysis system 70A, the possibility of the combination of the image data 301 and the patient identification information 320 leaking to the operator side can be reduced.

[0062] The acquisition unit 12 acquires the X-ray image data 300, the encrypted patient information 302, and the attribute information 303 sent from the user terminal 20, and sends them to the analysis unit 13.

[0063] The analysis unit 13 inputs the X-ray image data 300 sent from the acquisition unit 12 into the machine model 131, processes the output data from the machine model 131 as needed, and generates the analysis result data 173. The generated analysis result data 173 is at least associated with the encrypted patient information 302 and the attribute information 303, and is stored in the storage unit 17.

[0064] The communication control unit 11 acquires the analysis result data 173 associated with the encrypted patient information 302 and the attribute information 303 from the storage unit 17, and sends it to the user terminal 20 via the communication unit 15.

[0065] When the user terminal 20 receives the analysis result data 173, the encrypted patient information 302 is automatically decrypted to generate the patient identification information 320. The decryption of the encrypted patient information 302 can be performed when the analysis result data 173 is sent, or can be decrypted by the user himself.

[0066] Thus, the user terminal 20 can display the analysis result data 173 together with the patient identification information 320 including the patient's name or identification number on the screen.

[0067] Figure 7 It is a schematic diagram showing an example of the browser image 400 displayed on the screen of the user terminal 20 that has received the analysis result data 173. The browser image 400 is an example of the display image data generated by the generation unit 18. In the browser image 400, the text "<Bone density analysis>" indicating the content of the image analysis is displayed at the top. In the upper left part of the browser image 400, the text "<Analysis result>" indicating that it is the screen for displaying the analysis result is displayed. The patient's information can also be displayed below it. The text "Your bone density is □ g / cm" is displayed below it. 2Text like this. In box 401, the value of the estimated bone density is displayed. Further, text like “□% compared to a young person” can also be displayed below it. In box 402, the ratio relative to the bone density average of young adults (YAM, Young Adult Mean) is displayed. Text like “The probability of your femur fracture is □%” can also be displayed below it. In box 406, the probability of fracture is displayed in %. Further, text like “Judgment” can also be displayed below it, and text like “Osteopenia” etc. is displayed in box 403. For example, if the ratio relative to YAM is less than 80%, it is judged as “Osteopenia”, and if it is 70% or less, it is judged that there is a possibility of “Osteoporosis”. The browser image 400 may include an “End” button 404 and a “Return” button 405. The “Return” button can, for example, return to the previous screen, return to the home screen, or return to a given screen.

[0068] (Generation unit 18)

[0069] Next, the details of the generation unit 18 will be described. In the present embodiment, the generation unit 18 generates the derivation basis data 311 of the analysis result analyzed by the analysis unit 13. The communication control unit 11 may also include the derivation basis data 311 generated by the generation unit 18 and send it to the user terminal 20.

[0070] The analysis result data 173 is based on the output from the machine model 131, but the analysis process is not included in the output from the machine model 131. Therefore, usually, even if only the output from the machine model 131 is observed, it is impossible to judge the reliability of the output. Therefore, by sending the analysis result data 173 including the derivation basis data 311 to the user terminal 20, it is possible to help improve the user's confidence in the analysis result.

[0071] For example, the derived basis data 311 can also be processed image data 310 (basis image data) obtained by adding new information to the image data 301 input to the machine model 131. For example, the new information can also be information indicating the region that is the main basis for deriving the analysis result in the image data 301. The processed image can also be an image obtained by adding information indicating the region that is the main basis to the input image data. The information indicating the region is, for example, information on the range of the recognizable region such as coloring or bounding by a frame. In image analysis, in many cases, a certain region of the image becomes the main estimation basis. Therefore, by means of the information indicating such a region, the user can confirm the region that is the basis for estimation. In this case, the display image shown on the user terminal 20 includes a processed image obtained by adding region-of-interest information to the medical image input to the machine model 131. The generation unit 18 can also generate a display image in which the medical image (image data 301) and the processed image are juxtaposed. With such a display image, it is easy for the user to compare the original image that has been analyzed and the processed image. Alternatively, the control unit 16 can also switch between displaying the medical image and the processed image by a user operation. For example, the control unit 16 can alternately display the medical image and the processed image by the user clicking a switch button.

[0072] The processed image data 310 does not need to include all the information of the input image data 301. For example, the processed image data 310 can be image data obtained by cropping the input image data 301, or can also be an image with a lower resolution than the input image data 301. Thereby, the capacity of the data to be transmitted and received can be reduced.

[0073] Figure 8 It is a schematic diagram showing an example of the browser image 500 that sends the derived basis data 311 to the user terminal 20 in addition to the analysis result data 173. The browser image 500 is an example of the display image data generated by the generation unit 18. The browser image 500 adds the derived basis data 311 (including the processed image data 502 of the region 503) to the Figure 7 analysis result data 173 shown.

[0074] Specifically, in the browser image 500, together with the analysis result 501, the processed image data 502 obtained by adding a quadrilateral region 503 to the input image data ( Figure 6 X-ray image data 300) is displayed. As Figure 8As shown, in the present embodiment, the analysis result 501 derived from the machine model 131 that analyzes the image data and the processed image data 502 including the derivation basis data 311 are displayed on one screen. In the browser image 500, a "Return" button 506, an "End" button 507, and a "Predict the Future" button 505 can also be displayed. The function of the "Predict the Future" button 505 will be described later.

[0075] Figure 9 is Figure 8 An enlarged schematic view of the image of the region 503. Four lumbar vertebrae shown as L1 to L4 are included in the region 503. This indicates that the lumbar vertebrae L1 to L4 are the regions that are the basis for the analysis result. In fact, it is known that the bone density of the lumbar vertebrae L1 to L4 is related to the average value of the bone density of the whole body. That is, the region 503 surrounded by the frame represents the analysis result of the machine model 131 derived based on the bone density of the lumbar vertebrae L1 to L4.

[0076] The region of interest that is the basis for the analysis result may also include the region obtained by segmenting the medical image. The region of interest that is the basis for the analysis result may also include at least a part of the region obtained by segmenting the medical image. The region of interest that is the basis for the analysis result may also include the entire region obtained by segmenting the medical image. Segmentation means dividing the image into several regions. Segmentation is performed to reduce the analysis processing amount of the machine model 131. That is, the machine model 131 can analyze only the segmented region. The segmented region can be set to an arbitrary range. The segmented region can be, for example, a rectangle, a square, or a circle. When the segmented region is a square, the analysis processing amount of the machine model 131 can be reduced. In the case of analyzing an X-ray image of the lumbar vertebrae, for example, the range including the lumbar vertebrae L1 to L4 is segmented. Therefore, the size of the segmented region can also vary according to the size of the lumbar vertebrae L1 to L4 in the image. In the arrangement direction of the lumbar vertebrae L1 to L4, the segmented region can be set, for example, to be slightly larger than the lumbar vertebrae L1 to L4 or to overlap with the sides of the lumbar vertebrae L1 to L4. The segmented region can be, for example, always a given size or can be set according to the medical image. For example, the segmented region can also determine the positions of the lumbar vertebrae L1 to L4, and after setting the length in the arrangement direction of the lumbar vertebrae L1 to L4, set the length in the vertical direction of the lumbar vertebrae L1 to L4. Segmentation can also be performed by the machine model 131. To perform segmentation, the machine model 131 can learn the image with annotations of the analysis region.

[0077] In addition, the generation unit 18 may also generate a heat map of the region that serves as the basis for the analysis result. In this case, for example, the outer edge of the heat map points to the segmented region. A heat map is a method of representing the magnitude of bone density by the concentration of an arbitrary color. For example, the generation unit 18 may also generate a heat map representing the degree of concern. In addition, a heat map representing the numerical value of bone density may also be generated. In addition, the generation unit 18 may also generate a heat map representing the possibility (probability) of fracture. The processed image may also be an image in which a heat map representing the information of the region of interest is superimposed on the medical image. The image used for the heat map may be a static image or a dynamic image. By representing it with a dynamic image, for example, by gradually changing various heat maps in sequence, it is easy to visually recognize the relationship between the heat maps. In addition, in the case where the analysis result is a heat map including bone density outside the segmented region, a part of the segmented region may also be surrounded by a frame.

[0078] The generation unit 18 may also obtain the information of the region that serves as the basis for the analysis result from the analysis unit 13. Specifically, the generation unit 18 may obtain the region that serves as the basis for the analysis result from the analysis unit 13 and generate the information representing the region (such as the frame line surrounding the region 503). The generation unit 18 may also obtain the degree-of-concern data, bone density data, or information representing the possibility of fracture within the region from the analysis unit 13 and generate a heat map. The generation unit 18 may also generate any one or more of the heat map representing the degree-of-concern data, the heat map representing the bone density data, and the heat map representing the possibility of fracture. When the heat map representing the degree-of-concern data, the heat map representing the bone density data, and the heat map representing the possibility of fracture are overlapped and displayed, the generation unit 18 may also make the colors of the heat map representing the degree-of-concern data, the heat map representing the bone density data, and the heat map representing the possibility of fracture different.

[0079] The machine model 131 analyzes the X-ray image data 300, for example, through a neural network model (NNM, Neural Network Model). In the NNM, the following process is repeatedly performed: the image is temporarily segmented into small regions and quantized separately, and pooling processing is performed on multiple regions to integrate them into larger regions and quantize them. Therefore, the machine model 131 may also extract the region having a numerical value (such as a relatively large numerical value) that affects the processing result as the basis region.

[0080] In Figure 8In the example shown, the processed image data 502 in which the region 503 serving as the basis for the analysis result is superimposed and displayed on the X-ray image data 300 serving as the analysis target is described. However, the processed image data is not limited to this. For example, the analysis device 60A may transmit the position information (coordinates, etc.) of the region serving as the basis for the analysis result in the image serving as the analysis target to the user terminal 20, and cause the user terminal 20 to display the processed image data in which the region is displayed in the image serving as the analysis target.

[0081] Figure 8 The screen shown can be used when a doctor explains the analysis results to a patient. That is, the doctor can explain not only the analysis results to the patient, but also the region of the image data on which the analysis results appear. Therefore, the credibility of the doctor's analysis results can be improved, and the patient can also easily recognize the analysis results.

[0082] exist Figure 8 In the example shown, the analysis result 501 and the processed image data 502 including the derivation basis data 311 are displayed on one screen. However, "one screen" may not be displayed on the screen at the same time. The analysis result 501 and the processed image data 502 may be displayed by scrolling the screen up and down or left and right, for example. That is, the range in which the analysis result 501 and the processed image data 502 are displayed by scrolling the screen up and down or left and right is called "one screen".

[0083] Figure 10 This is a schematic diagram showing an example of a browser image 700 analyzed by the analysis unit 13, which displays the bone site of the patient's current fracture, the possibility of fracture at that site, and the possibility of fracture at that site in 3 years from now, estimated based on the image data. The future period is not limited to 3 years from now, but may be any period (for example, X years from now). The browser image 700 is an example of display image data generated by the generation unit 18. This can be displayed by clicking on the Figure 8 The "Predict the future" button 505 at the lower right of the browser image 500 shown in the figure is displayed. It is not limited to the future, and it can also be a prediction of a time different from the time point when the medical image was taken. Specifically, it can also be a period of time before the time point when the medical image was taken. In this case, "Predict the past" can be displayed instead of "Predict the future" or on the basis of "Predict the future".

[0084] The browser image 700 displays text such as <Bone Density Analysis> at the top, indicating the object of image analysis. At the upper left of the browser image 700, text such as <Predict the Future> is displayed, indicating that it shows a future prediction screen. Patient information can also be displayed below it. Text such as "The probability of your femur fracture is □%" is displayed below it. In the box 701, the value of the estimated probability of femur fracture at the current moment is displayed. Furthermore, text such as "The probability of femur fracture in 3 years is □%" can also be displayed below it. The box 702 displays the value of the predicted fracture probability in 3 years. In addition, information related to the estimation basis or prediction basis can also be displayed together in this image.

[0085] Figure 11 It means that in Figure 10 shown in the display image, an example of the browser image 800 with the export basis data added is illustrated. The browser image 800 is an example of the display image data generated by the generation unit 18. As Figure 11 shown, in the browser image 800, in addition to the area 801 showing the current moment and the probability of femur fracture in 3 years exported by the machine model 131, an image 802 representing the export basis data is also displayed.

[0086] Such a browser image 800 can be used by doctors to explain the current moment and future fracture probability to patients. When information related to the estimation basis or prediction basis is displayed together, the effect of increasing the persuasiveness to patients can be obtained.

[0087] (Machine model 131)

[0088] Next, the machine model 131 will be described. The machine model 131 is a model for estimating the state of the bones of the subject. The input image data includes images of bones, and the estimation result related to the state of the bones is output as the analysis result. For example, the machine model 131 is a learned model that is learned in the following way: based on the X-ray image of the bones, estimation results or calculation results related to the state of the bones, such as bone density, relative comparison of bone density, and probability of fracture, are output. The bone density can be the calculated bone density of the bone part included in the image data, or the average bone density of the whole body estimated from the image data. A well-known method can be used to calculate the bone density from the image. The relative comparison of bone density refers to the ratio of the estimated bone density to YAM. The probability of fracture refers to the probability that the bones at a specific part (such as the femoral neck, etc.) fracture. The estimation result related to the state of the bones can be the estimation result at the time when the image is taken, or the prediction at the time point after a given period has passed from that time point.

[0089] The machine model 131 can also output at least any one of the bone density estimated at the time when the image data is captured, the bone density predicted at the time point after a given period has elapsed since the time when the image data was captured, the fracture site estimated at the time when the image data was captured and its possibility, and the fracture site predicted at the time point after a given period has elapsed since the time when the image data was captured and its possibility as an estimation result.

[0090] (Learning method of the machine model 131)

[0091] Next, the learning method of the machine model 131 will be described. The learning of the machine model 131, for example, the learning of estimating the bone density, can also be performed using the X-ray image of the bone whose bone density has been determined as training data. The learning of estimating the possibility of fracture can also be performed using the X-ray image of the bone and the data on whether the patient has fractured within a given period thereafter as training data. The relative comparison of bone density does not require learning and is obtained by dividing the estimated bone density by YAM. In addition, the learning of future prediction can also be performed using the X-ray image of the bone whose bone density has been determined and the data on what degree of bone density the patient will have or whether a fracture has occurred after a given period as training data. By including the exercise amount, diet content, smoking, drinking, and other living habits of the patients who become the training data in the training data, a machine model 131 that can perform more accurate estimation or prediction can be constructed.

[0092] In the second embodiment, the machine model 131 is described as an example of a machine model that performs learning to estimate the state related to the bone such as the bone density. In this case, the input image data is the X-ray image of the bone, and the output is the estimation result related to the state of the bone. However, the machine model 131 is not limited to such a model. For example, the machine model 131 can also be a cytopathological analysis model. In this case, it can also be that the input medical image is a microscope image obtained by photographing the cells of the subject, and the analysis result is information related to the lesion of the cells.

[0093] Figure 12 It is a schematic diagram showing an example of a browser image 900 which is a display image that adds the cytopathological analysis result and its derived basis data when the machine model 131 is a cytopathological analysis model. The browser image 900 is an example of the display image data generated by the generation unit 18. In Figure 12In the browser image 900 shown, the text "Cytopathological Analysis", which represents the content of image analysis, is displayed at the topmost part. The text "<Judgment Result>", which indicates that it is a screen showing the result of determining whether the judged cell is benign or malignant, is displayed at the upper left part of the browser image 900. Information of the patient can also be displayed below it. The text "The possibility of malignancy is □%" is displayed in the lower area 901 below it. A numerical value representing the possibility that the cell is malignant is displayed in the square. Further, on the right side of the area 901, as the data for deriving the basis, the image 902 of the area 903, which is an area added to the analyzed image and serves as the main basis for deriving the judgment result, is displayed. The area 903 is an area including the cells judged to be malignant by the machine model 131. In addition, the browser image 900 can display an "End" button 904 and a "Return" button 905.

[0094] In addition, one screen can also be a screen displayed by scrolling the screen. Figure 13 It is a schematic diagram showing an example of displaying a display image on one screen by scrolling. Figure 13 The area 1301 shown by the solid line is the display area of the user terminal 20 (display device). The area 1302 shown by the dotted line is one screen. The data of the bone density analysis result 501 described in Figure 8 is displayed at the upper part of the area 1302. The processed image data 502, which is the data for deriving the basis as described in Figure 8 is displayed at the lower part of the area 1302. In the area 1301, when this area 1302 is initially displayed, as shown in the upper side of Figure 13 , the data of the bone density analysis result 501 is displayed. However, when the user scrolls the area 1302 upward, as shown in the upper side of Figure 13 , the processed image data 502 appears. In such a structure, the user can display the analysis result and the data for deriving its basis only by scrolling the screen.

[0095] According to the structure of the image remote analysis system 70A according to the above-described Embodiment 2, the data for deriving the basis 311 can be provided to the user together with the analysis result data 173. Therefore, the credibility of the analysis result for the user can be improved. In addition, when explaining to the patient, an effect that the patient can easily recognize the analysis result can be obtained.

[0096] Next, the process of the image remote analysis method S2 executed by the control unit 16 according to Embodiment 2 will be described. The image remote analysis method S2 includes steps S21 to S24 (not shown).

[0097] When the communication control unit 11 is accessed by the user to the user access analysis device 60A, the input screen for the acquisition information 172 including the image data 301 of the analysis object, the patient identification information 320, and the attribute information 303, that is, the browser image 200, is displayed on the web page via the communication unit 15 (S21).

[0098] The acquisition unit 12 acquires the encrypted patient information 302 obtained by encrypting the image data 301, the attribute information 303, and the patient identification information 320 included in the acquisition information 172 input to the browser image 200 and transmitted from the user terminal 20 (S22).

[0099] The analysis unit 13 analyzes the image data 301 transmitted from the acquisition unit 12 using the machine model 131 (S23).

[0100] The communication control unit 11 is a step of transmitting the analysis result data 173, the derivation basis data 311, and the encrypted patient information 302 analyzed by the analysis unit 13 to the user terminal 20 (S24).

[0101] According to the above image remote analysis method S2, the derivation basis data 311 can be provided to the user together with the analysis result data 173. Therefore, the credibility of the analysis result for the user can be improved. In addition, when explaining to the patient, the effect that the patient can easily recognize the analysis result can be obtained.

[0102] 〔Embodiment 3〕

[0103] Other embodiments of the present disclosure will be described below. In Embodiment 2, an analysis system for analyzing an X-ray image of a patient's bone and outputting the bone density, the relative comparison of the bone density, the possibility of fracture (%), etc. was described. However, in an image showing at least a part of the bone (for example, a simple X-ray image), sometimes not only the change in the image caused by the decrease in bone density but also the change in the image caused by other reasons may occur. As other reasons, there are certain diseases, treatment scars, various intentionally added information, items worn by the patient, etc. Certain diseases are, for example, arterial calcification, osteosclerosis, fractures, organ tumors, etc. Treatment scars are, for example, implants, bone cement, etc. Intentionally added information is, for example, letters such as "L" and "R" indicating directions. Items worn by the patient are, for example, necklaces, etc.

[0104] For these reasons, the brightness (whiteness) of the image changes more frequently compared to the case where there are no other reasons. In addition, implants, necklaces, etc. exhibit unique shapes. Such changes affect the evaluation of bone density by the analysis unit 13 of the analysis device 60A of the image remote analysis system 70A described in Embodiment 2. Usually, a doctor who views the image notices such changes and interprets the bone density evaluation result based on the information that causes such changes. However, depending on the situation, the doctor may not notice such changes in the image. Or, the doctor may also be unable to determine to what extent the analysis result of the analysis unit 13 takes into account the image changes caused by other reasons. In such a case, the doctor may hesitate how to interpret the analysis result of the analysis unit 13.

[0105] Therefore, it is desirable for the machine model to be able to not only analyze the object itself but also analyze the image changes caused by other reasons. The image remote analysis system 70B according to the present embodiment can execute the machine model to perform analysis related to the state of the object (e.g., bone) itself in the image, and detect image changes caused by other reasons, that is, reasons different from the state of the bone, and include at least any one of two analysis processes: analyzing including the change and analyzing excluding the change. Figure 14 It is a block diagram showing the structure of the image remote analysis system 70B according to Embodiment 3. The image remote analysis system 70B is the same as the image remote analysis system 70A according to Embodiment 2, and includes an analysis device 60B (image remote analysis device) for analyzing medical data (e.g., medical image data) and a user terminal 20. The analysis device 60B and the user terminal 20 are communicably connected via the Internet.

[0106] The analysis device 60B includes a communication unit 15, a control unit 16B, and a storage unit 17. The communication unit 15 and the storage unit 17 have the same functions as the communication unit 15 and the storage unit 17 described in Embodiment 2, so the description here is omitted. The control unit 16B includes a communication control unit 11, an acquisition unit 12, an analysis unit 13B, a generation unit (image generation unit) 18, and a determination unit 19. Each unit in the control unit 16B other than the analysis unit 13B has the same functions as the units described in Embodiment 2, so the description here is omitted.

[0107] The analysis unit 13B includes a machine model 131 and a second machine model 132. Similar to Embodiment 2, the machine model 131 is, for example, a machine model that analyzes the state of an object (such as a bone). In contrast, the second machine model 132 is a machine model learned in such a way that it detects changes in the image caused by other reasons, such as changes in the image caused by a certain disease, a treatment scar, various intentionally added information, an item worn by the patient, etc. The second machine model 132 can be learned using an image in which a doctor has annotated the positions of a certain disease, a treatment scar, various intentionally added information, an item worn by the patient, etc. The machine model 131 can include the changes in the image caused by other reasons detected by the second machine model 132 to perform an analysis related to the state of the object. In addition, the machine model 131 can remove the changes in the image caused by other reasons detected by the second machine model 132 to perform an analysis related to the state of the object.

[0108] The analysis unit 13B may also include a machine model that performs an analysis related to the state of a target object (such as a bone) itself, detects changes in the image caused by other reasons, and executes two analysis processes: an analysis process including the change and an analysis process removing the change.

[0109] Figure 15 This is an example of an image including the derived basis data of the image change based on other reasons generated by the generation unit 18 based on the analysis result analyzed by the second machine model 132. Figure 15 The area shown by the frame 1502 in [the figure] represents the area where the implant for fixing the spine is reflected. In addition, the area shown by the frame 1503 represents the area where the calcified abdominal aorta is reflected. In the figure, each area is represented by a dashed line, but it can also be represented by different colors such as a red frame and a yellow frame. The area shown by the frame 1501 is the same as that in Embodiment 2 and represents the area that is the basis for deriving the analysis result of the bone density of the lumbar vertebra that is the machine model 131. In this way, the range of the change in the image caused by other reasons can also be located outside the area represented by the frame 1501, which is the basis for deriving the result in the case of performing an analysis related to the state of the object.

[0110] The analysis unit 13B may also perform an analysis related to the state of the bone based on changes in the image for other reasons. In addition, the analysis unit 13B may remove the changes in the image caused by other reasons and perform an analysis related to the state of the bone. In addition, the generation unit 18 may generate analysis results in each case including the case of changes in the image for other reasons and the case of removal. The user may also specify which analysis method to use or which analysis result to display. In the case where there are multiple changing parts, it may be configured such that the user can separately specify which analysis method to use and which analysis result to display.

[0111] Figure 16 An example of the browser image 1600 is shown, in which analysis results in both the case including changes in the image due to other reasons and the case excluding the image changes are displayed. For example, the analysis image 1602 and the analysis result 1601 are displayed on the browser image 1600. The analysis image 1602 is the same as the Figure 15 image shown. In the analysis result 1601, the display of "In the case of the area including the dotted line, your bone density is XX g / cm 2 " and the display of "In the case of removing the area of the dotted line, your bone density is YY g / cm 2 " are implemented. At the lower part of the analysis image 1602, there is a selection button 1606. By clicking this button, it is possible to move to a screen where one or both of the areas 1604 and 1605 including the dotted line can be selected for analysis and which analysis result to display, etc. The return button 1607 and the end button 1608 are as described above.

[0112] As described above, an example of the image remote analysis system 70B evaluating the bone density of the femur or the spine has been described. However, the object to be evaluated is not limited to this. For example, the image remote analysis system 70B can evaluate the X-ray image of the bones in the chest and can also analyze the images of objects other than bones.

[0113] Figure 16 An example of the browser image 1600 is shown, in which analysis results in both the case including changes in the image due to other reasons and the case excluding the image changes are displayed. For example, the analysis image 1602 and the analysis result 1601 are displayed on the browser image 1600. The analysis image 1602 is the same as the Figure 15 image shown. In the analysis result 1601, the display of "In the case of the area including the dotted line, your bone density is XX g / cm 2 " and the display of "In the case of removing the area of the dotted line, your bone density is YY g / cm 2The display of "". At the lower part of the analyzed image 1602, there is a selection button 1606. By clicking this button, it is possible to move to a screen where one or both of the areas 1604 and 1605 including the dotted line can be selected for analysis and the display of which analysis result, etc. The return button 1607 and the end button 1608 are as described above.

[0114] 〔Implementation Example Using Software〕

[0115] The functions of the image remote analysis system 70A (hereinafter referred to as "the system") are programs for enabling a computer to function as the system, and can be implemented by programs for enabling each part of the computer to function as the system.

[0116] In this case, the above system includes a computer having at least one control device (such as a processor) and at least one storage device (such as a memory) as the hardware for executing the above programs. By using the control device and the storage device to execute the above programs, the various functions described in the above embodiments are realized.

[0117] The above programs can also be recorded in one or more non-temporary, computer-readable recording media. This recording medium may or may not be provided by the above device. In the latter case, the above programs can also be supplied to the above device via any wired or wireless transmission medium.

[0118] In addition, part or all of the functions of the above parts can also be realized by logic circuits. For example, an integrated circuit formed with a logic circuit for each of the above parts also falls within the scope of the present disclosure. In addition to this, for example, the functions of the above parts can also be realized by a quantum computer.

[0119] As described above, the invention related to the present disclosure has been described based on the respective drawings and embodiments. However, the invention related to the present disclosure is not limited to the above embodiments. That is, the invention related to the present disclosure can be variously modified within the scope shown in the present disclosure, and embodiments obtained by appropriately combining the technical means separately disclosed in different embodiments are also included in the technical scope of the invention related to the present disclosure. That is, it should be noted that those skilled in the art can easily make various deformations or corrections based on the present disclosure. In addition, please note that these deformations or corrections are included within the scope of the present disclosure.

[0120] In the present disclosure, the analysis of medical images has been described, and the medical images are changed based on region-of-interest information related to the region of interest that is focused on during the process of outputting the analysis result. However, the present disclosure is not limited thereto. For example, instead of including medical images, only numerical values and data of the information shown below can be used as input information to analyze using a learned machine model. In this case, the region-of-interest information that is focused on during the process of outputting the analysis result can be displayed on the display image together with the analysis result. In addition, the region-of-interest information can also emphasize a part of the input information. For example, when the information of age and gender is the region of interest, the items or values of age and gender can be highlighted in a conspicuous color, etc.

[0121] The medical image can also be, for example, an image obtained by photographing a subject with an endoscope. More specifically, the medical image includes an image obtained by photographing a part including at least one of the nasal cavity, esophagus, stomach, duodenum, rectum, large intestine, small intestine, anus, and colon of the subject with an endoscope. The medical image of these parts being photographed can, for example, output an analysis result that clearly shows a region of interest including at least one of inflammation, polyps, and cancer through a learning model. In such a case, as the learning model, for example, a model learned based on a first learning image including an image with a region of interest and first training data indicating the presence of the region of interest, a second learning image including an image without a region of interest, and second training data indicating the absence of the region of interest can be used. The first training data can also include information indicating the degree of inflammation (the degree of inflammation) or the degree of malignancy (the degree of malignancy) of the region of interest. The analysis result can be, for example, a display surrounding the region of interest, a display indicating the region of interest, or a display with a color overlapping the region of interest. The derivation basis data indicating the basis for deriving the analysis result can also be displayed together with such an analysis result in the same manner as in the above-described embodiment.

[0122] Alternatively, a medical image can also be, for example, an image obtained by photographing a subject's eye, skin, etc. with a digital camera. A medical image of these parts can also output, for example, an analysis result that explicitly indicates a symptom of concern through a learning model. For example, if it is an eye, the symptoms of concern can also include symptoms of diseases indicating at least one of glaucoma, cataract, age-related macular degeneration, conjunctivitis, hordeolum, retinopathy, and blepharitis. Alternatively, as long as it is the skin, the symptoms of concern can include, for example, symptoms of skin cancer, urticaria, atopic dermatitis, and herpes. The analysis result can be a display surrounding these symptoms of concern, a display indicating the symptoms of concern, a display with a color superimposed on the symptoms of concern, or the name of the disease can also be displayed. As a learning model, for example, a model can be used that is learned based on first learning images including images of these parts and first training data indicating the presence of symptoms of concern, second learning images including images without symptoms of concern, and second training data indicating the absence of symptoms of concern. The export basis data indicating the basis for deriving the analysis result can also be displayed together with such an analysis result in the same manner as in the above-described embodiments.

[0123] Examples of input information include age, gender, weight, height, presence or absence of fractures, fracture site, fracture history, fracture history of family members (e.g., parents), underlying diseases (e.g., food and / or drug allergies, diseases related to the onset of osteoporosis and unrelated diseases, etc.), smoking history, drinking habits (e.g., drinking frequency and amount), occupation history, exercise history, medical history (e.g., medical history of bone diseases), menstruation (e.g., cycle and presence or absence), menopause (e.g., possibility and presence or absence), artificial joints (e.g., type, presence or absence, and period of replacement surgery of spinal implants or knee joints, etc.), results of blood tests, results of urine tests, medications taken, gene sequences, etc.

[0124] (Summary)

[0125] (Mode 1)

[0126] The image generation device according to Mode 1 of the present disclosure includes: an acquisition unit that acquires an analysis result output from a machine model that analyzes a medical image obtained from a subject, and region-of-interest information related to a region of interest within the medical image that is of concern in the process of outputting the analysis result; and an image generation unit that generates a display image obtained by changing the medical image based on the region-of-interest information.

[0127] (Mode 2)

[0128] In the image generation device according to Mode 2 of the present disclosure, in the above Mode 1, the display image includes a processed image obtained by adding the region-of-interest information to the medical image input to the machine model.

[0129] (Mode 3)

[0130] For the image generation device according to Mode 3 of the present disclosure, in the above Mode 1 or 2, the region of interest includes the region obtained by segmenting the medical image.

[0131] (Mode 4)

[0132] For the image generation device according to Mode 4 of the present disclosure, in any one of the above Modes 1 to 3, the region of interest information is represented by a heat map.

[0133] (Mode 5)

[0134] For the image generation device according to Mode 5 of the present disclosure, in the above Mode 2, the processed image is an image on which the region of interest information is superimposed.

[0135] (Mode 6)

[0136] For the image generation device according to Mode 6 of the present disclosure, in the above Mode 2 or 5, the image generation unit generates the display image in which the medical image and the processed image are juxtaposed.

[0137] (Mode 7)

[0138] For the image generation device according to Mode 7 of the present disclosure, in any one of the above Modes 2, 5, and 6, the display image and the processed image can be replaced by an operation.

[0139] (Mode 8)

[0140] For the image generation device according to Mode 8 of the present disclosure, in any one of the above Modes 1 to 7, the machine model is a model for estimating the state of the bones of the subject, the medical image is an image showing the bones of the subject, and the analysis result includes an estimation result related to the state of the bones of the subject.

[0141] (Mode 9)

[0142] For the image generation device according to Mode 9 of the present disclosure, in any one of the above Modes 1 to 8, the display image represents the analysis result and the region of interest information by heat maps in different ways.

[0143] (Mode 10)

[0144] For the image generation device according to Mode 10 of the present disclosure, in the above Mode 8, the estimation result is at least any one of the result of estimating the fracture site of the subject and the possibility of the subject having a fracture.

[0145] (Mode 11)

[0146] In the image generation device according to Mode 11 of the present disclosure, in any one of the above Modes 1 to 7, the machine model is a cytopathological analysis model, the medical image is a microscopic image obtained by photographing the cells of the subject, and the analysis result includes information related to the lesions of the cells of the subject.

[0147] (Mode 12)

[0148] In the image generation device according to Mode 12 of the present disclosure, in any one of the above Modes 1 to 11, the display image is a display image displayed on one screen.

[0149] (Mode 13)

[0150] In the image generation device according to Mode 13 of the present disclosure, in the above Mode 1, the machine model detects changes in the image caused by reasons different from the state of the object, and performs at least any one of the following two analysis processes: analyzing including the change; analyzing after removing the change.

[0151] (Mode 14)

[0152] In the image generation device according to Mode 14 of the present disclosure, in the above Mode 13, the machine model includes: a machine model for estimating the state of the object itself, and a second machine model for detecting changes in the image caused by reasons different from the state of the object.

[0153] (Mode 15)

[0154] In the image generation device according to Mode 15 of the present disclosure, in the above Mode 13 or 14, it is possible to display at least any one of the result of analyzing including the change in the image and the result of analyzing after removing the change.

[0155] (Mode 16)

[0156] The image generation method according to Mode 16 of the present disclosure includes: an acquisition step of acquiring an analysis result output from a machine model that analyzes a medical image obtained by photographing a subject, and attention area information related to an attention area within the medical image and that is concerned during the output of the analysis result; and an image generation step of generating a display image that changes the medical image based on the attention area information.

[0157] (Mode 17)

[0158] The program related to Mode 17 of the present disclosure is an image generation program for causing a computer to function as the image generation device described in Mode 1, and is an image generation program for causing a computer to function as the acquisition unit and the image generation unit described above.

[0159] (Mode 18)

[0160] The recording medium related to Mode 18 of the present disclosure is a computer-readable non-transitory recording medium that records the image generation program described in Mode 17.

[0161] (Mode 19)

[0162] The display device of Solution 19 of the present disclosure displays an analysis result and a processed image. The analysis result is output from a machine model that analyzes a medical image obtained from a subject, and the processed image is obtained by processing the medical image based on region-of-interest information related to a region of interest, where the region of interest is a region within the medical image and is of concern during the output of the analysis result.

[0163] -Symbol Explanation-

[0164] 70A, 70B... Image remote analysis system

[0165] 3... Image generation device

[0166] 11, 33... Communication control unit

[0167] 12... Acquisition unit

[0168] 13... Analysis unit

[0169] 131... Machine model

[0170] 14... Analysis result sending unit

[0171] 15, 50... Communication unit

[0172] 16, 30... Control unit

[0173] 17, 40... Storage unit

[0174] 18... Generation unit

[0175] 19... Judgment unit

[0176] 20... User terminal

[0177] 31... Image generation unit

[0178] 32... Data acquisition unit (acquisition unit)

[0179] 41... Analysis data

[0180] 42... Derivation basis data

[0181] 43... Display image data

[0182] 60, 60A, 60B... Analysis device

[0183] 601... Machine model

[0184] 70... Display device.

Claims

1. An image generation device, comprising: an acquisition unit that acquires an analysis result output from a machine model that analyzes a medical image obtained from a subject, and derivation basis data indicating the basis for deriving the analysis result; and an image generation unit that generates a display image obtained by changing the medical image based on the derivation basis data.

2. The image generation device according to claim 1, wherein, the display image includes a processed image obtained by adding the derivation basis data to the medical image input to the machine model.

3. The image generation device according to claim 1 or 2, wherein, the derivation basis data, i.e., the region of interest, includes regions obtained by segmenting the medical image.

4. The image generation device according to any one of claims 1 to 3, wherein, the derivation basis data is represented as a heat map.

5. The image generation device according to claim 2, wherein, the processed image is an image obtained by superimposing the derivation basis data.

6. The image generation device according to claim 2, wherein, the image generation unit generates the display image in which the medical image and the processed image are arranged side by side.

7. The image generation device according to claim 2, wherein, the medical image and the processed image can be replaced by an operation.

8. The image generation device according to any one of claims 1 to 7, wherein, the machine model is a model for estimating the state of the subject's bones, the medical image is an image showing the subject's bones, the analysis result includes an estimation result related to the state of the subject's bones.

9. The image generation device according to claim 8, wherein, the display image represents the analysis result and the derivation basis data as heat maps in different ways.

10. The image generation device according to claim 8 or 9, wherein, the estimation result is at least one of a result of estimating the subject's fracture site and the possibility of the subject having a fracture.

11. The image generation device according to any one of claims 1 to 7, wherein, the machine model is a cytopathological analysis model, the medical image is a microscopic image obtained by photographing the subject's cells, the analysis result includes information related to the lesion of the subject's cells.

12. The image generation device according to any one of claims 1 to 11, wherein, the display image is a display image displayed on one screen.

13. The image generation device according to any one of claims 1 to 12, wherein, the machine model detects changes in an image caused by reasons different from the state of the object, and performs at least one of two analysis processes including analyzing the change and analyzing after removing the change.

14. The image generation device according to claim 13, wherein, the machine model includes a machine model for estimating the state of the object itself and a second machine model for detecting changes in an image caused by reasons different from the state of the object.

15. The image generation device according to claim 13 or 14, wherein it is capable of displaying at least any one of a result obtained by analyzing a change in the image and a result obtained by analyzing the removal of the change.

16. An image generation method, comprising the following steps: an acquisition step of acquiring an analysis result output from a machine model that analyzes a medical image of a subject and derivation basis data indicating a basis for deriving the analysis result; and an image generation step of generating a display image obtained by changing the medical image based on the derivation basis data.

17. An image generation program for causing a computer to function as the image generation device according to claim 1, for causing the computer to function as the acquisition unit and the image generation unit described above.

18. A computer-readable non-transitory recording medium, on which the image generation program according to claim 17 is recorded.

19. A display device that displays: an analysis result output from a machine model that analyzes a medical image of a subject; and a processed image obtained by processing the medical image based on region-of-interest information related to a region of interest, the region of interest being a region within the medical image and being of interest during the output of the analysis result.

Images