Image generation apparatus, image generation method, display device, image generation program, and recording medium

Abstract

To further improve confidence in analysis results obtained using machine models. [Solution] The image generation device comprises an acquisition unit that acquires analysis results output from a machine model that analyzes medical images of a subject, and derivation basis data that shows the basis for the derivation of the analysis results, and an image generation unit that generates a display image by modifying the medical image based on the derivation basis data.

Description

Technical Field

[0001] The present disclosure relates to an image generation device, an image generation method, a display device, an image generation program, and a recording medium.

Background Art

[0002] For example, Patent Document 1 discloses an X-ray age estimation model that indicates the correspondence between a feature value obtained from X-ray image data and the age of a subject, using the X-ray image data of the subject and information on the subject's age at the time the X-ray image was captured.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, analysis of various images using trained machine models has been performed. However, the grounds for the analysis results obtained using machine models are often unclear. An object of the present disclosure is to further improve the reliability in the analysis results obtained using a machine model.

Means for Solving the Problems

[0005] An image generation device according to one aspect of the present disclosure includes an acquisition unit that acquires an analysis result output from a machine model that analyzes a medical image obtained by imaging a subject, and derivation basis data indicating the basis on which the analysis result was derived, and an image generation unit that generates a display image obtained by modifying the medical image based on the derivation basis data.

[0006] An image generation method according to one aspect of the present disclosure includes an acquisition step of acquiring analysis results output from a machine model that analyzes medical images of a subject and derivation basis data indicating the basis for which the analysis results were derived, and an image generation step of generating a display image by modifying the medical image based on the derivation basis data.

[0007] Each aspect of the present disclosure may be implemented by a computer, in which case a control program for the image generation device that enables the computer to implement the image generation device by operating the computer as each part (software element) of the image generation device, and a computer-readable recording medium on which the program is recorded, also fall within the scope of the present disclosure. [Effects of the Invention]

[0008] According to one aspect of this disclosure, confidence in the analysis results obtained using a machine model can be further improved. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the configuration of an image generation device according to Embodiment 1 of this disclosure. [Figure 2] This is a flowchart showing the flow of the image display method according to Embodiment 1. [Figure 3] This is a schematic diagram showing an example of a display image according to Embodiment 1. [Figure 4] This is a block diagram showing the configuration of an image remote analysis system according to Embodiment 2 of this disclosure. [Figure 5] This is a schematic diagram showing an example of a browser image for user input. [Figure 6] This is a schematic diagram showing an example of X-ray image data used for bone density analysis. [Figure 7] This is a schematic diagram showing an example of a browser image of the analysis results transmitted by the analysis result transmission unit. [Figure 8] Figure 7 is a schematic diagram showing an example of a browser image with the derivation data added to the displayed image. [Figure 9] Figure 8 is a magnified schematic diagram of the image of the region shown. [Figure 10] This is a schematic diagram showing an example of a browser image that displays the current and future risk of fractures in patients. [Figure 11] This is a schematic diagram showing an example of a browser image with the derivation data added to the display image shown in Figure 10. [Figure 12] This is a schematic diagram showing an example of a display image with added data for the derivation of cellular pathology analysis results. [Figure 13] This is a schematic diagram illustrating an example of scrolling to display images on a single screen. [Figure 14] This is a block diagram showing the configuration of the image remote analysis system according to Embodiment 3. [Figure 15] This is an example of an image that includes data supporting the derivation of image changes due to other reasons, as analyzed by a machine model. [Figure 16] This is an example of a browser image showing the analysis results for both cases: with and without image changes due to other reasons. [Modes for carrying out the invention]

[0010] [Embodiment 1] Hereinafter, an embodiment of the present disclosure will be described in detail with reference to the drawings. Figure 1 is a block diagram showing the configuration of an image generation device 3 according to Embodiment 1 of the present disclosure. The image generation device 3 is a device that generates a display image by modifying a medical image to include the basis for the analysis result, based on focus region information related to a focus region that was identified during the process in which an analysis result is output from a machine model that analyzes a medical image. The image generation device 3 can communicate information with an analysis device 60 and a display device 70 via a communication unit 50. The analysis device 60 includes a machine model 601. The machine model 601 is, for example, a trained machine model that analyzes at least one of an image and a numerical value. The display device 70 displays, for example, the results analyzed by the machine model 601.

[0011] As shown in Figure 1, the image generation device 3 includes a control unit 30, a storage unit 40, and a communication unit 50. The control unit 30 includes an image generation unit 31, a data acquisition unit (acquisition unit) 32, and a communication control unit 33.

[0012] The data acquisition unit 32 acquires the analysis results (analysis data) output (derived) from the machine model 601 that analyzes the image, and the data for which the analysis results are derived. The data for which the results are derived is, for example, information on a region of interest within the medical image that was focused on during the process of outputting the analysis results. The data acquisition unit 32 records the acquired analysis data 41 and the data for which the results are derived in the storage unit 40. The medical image may include, for example, at least one of the following: a plain X-ray image, a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, a PET (Positron Emission Tomography) image, and an ultrasound image. When a plain X-ray image is used as a medical image, the area of ​​the plain X-ray image is not particularly limited. For example, a plain X-ray image used as a medical image may show a part of the subject's skeleton. More specifically, a simple X-ray image only needs to show at least one of the following parts of the subject: head, neck, chest, waist, hip joint, knee joint, ankle joint, foot, toes, shoulder joint, elbow joint, wrist joint, hand, fingers, or temporomandibular joint. Estimation simple X-ray images are used to estimate the bone density of the subject's bones and the location of fractures. Such estimation simple X-ray images may be frontal images (for example, images obtained by irradiating the subject with X-rays in the anterior-posterior direction) or lateral images (for example, images obtained by irradiating the subject with X-rays in the lateral direction). When using CT images as medical images, at least one of the following can be used: a three-dimensional image, a cross-sectional image perpendicular to the body axis connecting the head and legs (for example, a horizontal section), and a cross-sectional image parallel to the body axis (for example, a sagittal or coronal section). Medical images may show bones. Medical images do not need to show bone. Bone density may be expressed by at least one of the following: bone mineral density per unit area (g / cm2), bone mineral density per unit volume (g / cm3), YAM (%), T-score, and Z-score. YAM (%) is an abbreviation for "Young Adult Mean" and is sometimes called the average percentage for young adults.For example, the bone mineral density of a bone may be a value expressed in terms of bone mineral density per unit area (g / cm²) and YAM (%). The bone mineral density of a bone may be an index defined in guidelines, or may be a proprietary index.

[0013] The image generation unit 31 generates a display image (display image data) obtained by modifying the medical image based on the region-of-interest information. The image generation unit 31 records the generated display image data 43 in the storage unit 40.

[0014] The communication control unit 33 transmits the display image data 43 recorded in the storage unit 40 to the display device 70. In this case, the communication control unit 33 may transmit the display image data 43 to the display device 70. For example, the communication control unit 33 may transmit the image data used for analysis first, and then transmit the analysis data 41 thereafter. Alternatively, for example, the communication control unit 33 may transmit the analysis data 41 first, and then transmit the image data used for analysis thereafter.

[0015] The control unit 30 performs overall control of the entire image generation device 3. The control unit 30 includes at least one processor and at least one memory. The processor can be configured using, for example, at least one general-purpose processor such as an MPU (Micro Processing Unit) or a CPU (Central Processing Unit). The memory may include multiple types of memory such as ROM (Read Only Memory) and RAM (Random Access Memory). As an example, the processor implements the functions of the image generation device 3 by loading various control programs recorded in the ROM of the memory into the RAM and executing them. Further, the processor may include a processor configured with an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a PLD (Programmable Logic Device), or the like.

[0016] Figure 2 is a flowchart illustrating the flow of an image display method S1 using an image generation device 3 as an example. As shown in the figure, the image display method S1 includes steps S11 to S13.

[0017] The data acquisition unit 32 acquires the analysis results output from the machine model 601 that analyzes medical images of the subject, and derivation basis data that shows the basis for the derivation of the analysis results (S11). Specifically, for example, the data acquisition unit 32 acquires the analysis results output from the machine model that analyzes medical images of the subject, and focus region information relating to a region within the medical image that was focused on during the process of outputting the analysis results. The region within the medical image may be, for example, a region inside the outer edge of the medical image. The focus region information may be entirely located within the region of the medical image, or only partially overlap. Alternatively, if the focus region information is not present in the medical image (or, for example, the entire medical image), it does not need to overlap with a region within the medical image. In this case, for example, a statement indicating that there is no focus region (or, for example, that the entire medical image is the focus region) may be displayed outside the region within the medical image. The following explanation uses the example of a human being (i.e., "subject"), but the subject is not limited to humans. The subject may be a mammal other than a human, such as a horse, feline, canine, bovine, or pig. Furthermore, this disclosure includes embodiments in which "subject" is replaced with "animal" if the embodiment is applicable to these animals as well.

[0018] The image generation unit 31 generates a display image by modifying the medical image based on the acquired derived basis data (S12). Specifically, the image generation unit 31 generates a display image by modifying the medical image based, for example, on the area of ​​focus information.

[0019] The communication control unit 33 transmits the display image generated by the image generation unit 31 to the display device 70 (step S13). Through this processing method, the analysis results derived by the machine model that analyzes the image and the data on which the analysis results were derived can be displayed on a single screen. Using this method, as an example, a display image 100 is generated and displayed on the display device 70, which displays on a single screen the analysis results output from the machine model that analyzes medical images of a subject, as shown in Figure 3, and the area of ​​focus information relating to the area of ​​focus that was focused on during the process of outputting the analysis results within the medical image.

[0020] Figure 3 is a schematic diagram showing an example of a display image (display screen) 100 according to Embodiment 1. Specifically, Figure 3 is a display image in which the analysis results derived by a machine model that analyzes an image and the data on which the analysis results were derived are displayed on a single screen. The data on which the analysis results were derived refers to at least one piece of evidence explaining why the machine model derived such an analysis result. For example, the data on which the analysis results were derived may be information indicating which part of the image (medical image) was used as the basis for deriving the analysis result, or it may be information indicating which part was given what weight in the derivation. The display image is an image displayed on the screen (display screen) of a display device. The display image may be a browser image, that is, an image displayed on a website established on the internet that can be viewed using a browser. The type of display device is not limited. The screen may be, for example, the screen of a stationary personal computer or the screen of a mobile terminal.

[0021] Figure 3 specifically shows the analysis results derived by the machine model 601, which analyzes images containing bone, and the data supporting the derivation of those analysis results, all displayed on a single screen (the screen displaying the image 100). The analysis results are displayed in region 101 and include bone density and the percentage of that bone density compared to the average bone density of young adults. The data supporting the derivation of those analysis results is displayed in region 102. Region 102 displays an image containing bone analyzed by the machine model 601, with the region of interest 103 added. In this embodiment, the region of interest is the region that served as the main basis for deriving the image analysis results. The "Future Prediction" button 105, the "Back" button 106, and the "Exit" button 107 will be described later.

[0022] As shown in Figure 3, the analysis results derived by the machine model 601 that analyzes the image, and the data supporting the derivation of those analysis results, can be displayed on a single screen. This allows the user to see at least one piece of evidence explaining why the machine model derived those results, along with the analysis results themselves.

[0023] By using the image generation device 3 or image display method S1 described above, the analysis results derived by the machine model that analyzes the image, and the data supporting the derivation of those analysis results, can be displayed on a single screen. This allows the user to confirm, along with the analysis results, one of the reasons why the machine model derived those results. Therefore, compared to displaying only the analysis results, the user's confidence in the analysis results obtained using the machine model can be further enhanced.

[0024] [Embodiment 2] Other embodiments of the present disclosure are described below. In this embodiment, an example of a displayed image will be described, taking as an example a case in which an analysis system (also called an image remote analysis system) 70A, which receives an image analysis request from a user, analyzes an X-ray image of bone and outputs bone density, relative comparison of bone densities, fracture probability (%), etc. The image remote analysis system 70A can display to the user, along with the analysis results, one of the reasons why the machine model derived such analysis results.

[0025] (Configuration of Image Remote Analysis System 70A) Figure 4 is a block diagram showing the configuration of the image remote analysis system 70A according to Embodiment 2 of the present disclosure. As shown in the figure, the image remote analysis system 70A comprises an analysis device 60A (image remote analysis device) for analyzing medical data (e.g., medical image data) and a user terminal 20, and the analysis device 60A and the user terminal 20 are connected to each other via the Internet.

[0026] The analysis device 60A comprises a communication unit 15, a control unit 16, and a storage unit 17. The communication unit 15 communicates with the user terminal 20 via the internet. The storage unit 17 stores acquired information 172, which includes image data 301, encrypted patient information 302, and attribute information 303, and analysis result data 173, which includes processed image data 310 and derivation basis data 311. The storage unit 17 stores the image data 301, encrypted patient information 302, and attribute information 303 acquired by the acquisition unit 12, linking them together.

[0027] The control unit 16 comprises a communication control unit 11 (analysis result transmission unit), an acquisition unit 12, an analysis unit 13, a generation unit (image generation unit) 18, and a determination unit 19. The communication control unit 11 controls the communication unit 15. The acquisition unit 12 acquires data transmitted from the user terminal 20 via the communication control unit 11. The acquisition unit 12 also acquires analysis results (analysis data) derived by the analysis unit 13 (machine model 131) which analyzes medical images, and the data on the basis for the derivation of said analysis results. The analysis unit 13 analyzes image data 301. The control unit 16 may have the same configuration as the control unit 30 described in Embodiment 1.

[0028] The generation unit 18 generates the derivation basis data for the analysis results analyzed by the analysis unit 13. The derivation basis data 311 means at least one reason why the machine model derived such an analysis result. For example, the derivation basis data 311 may be information indicating which part of the image was primarily used to derive the analysis result. The derivation basis data 311 is linked to at least the encrypted patient information 302 and stored in the storage unit 17. Specific examples of the functions of the generation unit 18 will be described later. The generation unit 18 also generates a display image (display image data) in which the analysis results acquired by the acquisition unit 12 and the area of ​​interest information (derivation basis data for the analysis results) are displayed on one screen. The generation unit 18 records the generated display image data in the storage unit 17.

[0029] (Judgment section 19) The determination unit 19 determines whether the encrypted patient information 302 acquired by the acquisition unit 12 is actually encrypted patient identification information 320. If it is determined that the encrypted patient information 302 is not actually encrypted, the acquisition unit 12 may delete the acquired encrypted patient information 302. By providing such a determination unit 19, if the encrypted patient information 302 transmitted from the user terminal 20 is not encrypted patient identification information 320, the acquired image data 301 is deleted along with the encrypted patient information 302 and attribute information 303 without being analyzed. Therefore, the acquired information 172, including the image data 301, is not stored inside the analysis device 60A. The determination of whether or not something is encrypted may be made, for example, from the file extension of the encrypted data.

[0030] If it is determined that the encrypted patient information 302 is not actually encrypted, the control unit 16 may send a message to the user terminal 20 via the communication control unit 11 requesting that the patient identification information 320 be encrypted and sent again.

[0031] (Browser image 200) Figure 5 is a schematic diagram showing an example of a browser image 200 that is displayed on a web page when a user accesses the analysis device 60A via the internet, for inputting image data 301, etc.

[0032] As shown in the diagram, the browser image 200 displays the text "<Bone Density Analysis>" at the top, indicating the subject of the image analysis. The text displayed at the top is not limited to this; for example, it could be any indication of the analysis content, such as "<Bone Density Estimation>" or "<Bone Density Future Prediction>". The upper left of the browser image 200 may contain status information indicating the current phase or content of the analysis. For example, the upper left of the browser image 200 could, but is not limited to, "<Reception>", "<Analysis Results>", "<Future Prediction>", or "<Benign / Malignant Determination>".

[0033] Furthermore, in the upper left corner of browser image 200, the text "<Reception>" is displayed, for example, to indicate that it is a screen for accepting input. Below that, the text "Please place the image to be analyzed here" prompts the user to input the image data to be analyzed, and a frame 205 is displayed indicating the area for pasting (dragging and dropping) the image data. Further below that, the text "Please enter the attribute information here" prompts the user to input patient identification information 320 and attribute information 303, and an input box 206 is displayed. In the lower right corner of browser image 200, a "Submit" button 204 prompts the user to submit, and in the upper right corner, a "Back" button 207 is displayed to return to the initial screen of the website.

[0034] Figure 6 is a schematic diagram showing X-ray image data 300 for which bone density analysis is requested, as an example of image data 301. The X-ray image data 300 may be simple X-ray images such as lumbar X-ray images or chest X-ray images, or it may be an X-ray image taken with a DXA (Dual energy X-ray Absorptiometry) device. In a DXA device that measures bone density using the DXA method, when measuring bone density of the lumbar spine, X-rays are irradiated from the front of the subject's lumbar spine. Also, in a DXA device, when measuring bone density of the proximal femur, X-rays are irradiated from the front of the subject's proximal femur. The ultrasound method is a method of measuring bone density by applying ultrasound to bones such as the heel and tibia. Here, "front of the lumbar spine" and "front of the proximal femur" refer to the direction that correctly faces the imaging site such as the lumbar spine and proximal femur, and may be the ventral side of the subject's body or the posterior side of the subject's body. Microdensitometry (MD) involves irradiating the hand with X-rays. Ultrasound is a method of measuring bone density by applying ultrasound to bones such as the lumbar spine, femur, heel, or tibia. Furthermore, the image data does not have to be an X-ray image; any image containing bone information is acceptable. Image data can be estimated from, for example, MRI (magnetic resonance imaging) images, CT (computed tomography) images, PET images, and ultrasound images.

[0035] In this embodiment, when a user accesses the analysis device 60A via the Internet, the communication control unit 11 may display a browser image 200 on a web page and send an encryption application via the communication unit 15. When the user enters the image data 301, patient identification information 320, and attribute information 303 to be analyzed and clicks the "Send" button 204, the patient identification information 320 is encrypted, and the encrypted patient information 302, along with the image data 301 and attribute information 303, is sent to the analysis device 60A. Therefore, the user does not need to perform the encryption process for the patient identification information 320. The encryption process may be performed after the user clicks the "Send" button 204, rather than when the user clicks it.

[0036] The encryption method may be any known method and is not limited to it. For example, the encryption method may be an encryption method that combines a public key and a private key. The encryption may be such that even the operator of the remote image analysis system 70A cannot decrypt it. This reduces the risk that the combination of image data 301 and patient identification information 320 may be leaked to the operator even if the user requests image analysis from the operator of the remote image analysis system 70A.

[0037] The acquisition unit 12 acquires the X-ray image data 300, encrypted patient information 302, and attribute information 303 transmitted from the user terminal 20, and transmits them to the analysis unit 13.

[0038] The analysis unit 13 inputs the X-ray image data 300 transmitted from the acquisition unit 12 to the machine model 131, processes the output data from the machine model 131 as needed, and generates analysis result data 173. The generated analysis result data 173 is linked to at least encrypted patient information 302 and attribute information 303 and stored in the storage unit 17.

[0039] The communication control unit 11 acquires analysis result data 173 linked to encrypted patient information 302 and attribute information 303 from the storage unit 17 and transmits it to the user terminal 20 via the communication unit 15.

[0040] When the user terminal 20 receives the analysis result data 173, the encrypted patient information 302 is automatically decrypted and patient identification information 320 is generated. The encrypted patient information 302 may be decrypted when the analysis result data 173 is transmitted, or the user may decrypt it themselves.

[0041] As a result, the user terminal 20 can display the analysis result data 173 on its screen along with patient identification information 320, which includes the patient's name or identification number.

[0042] Figure 7 is a schematic diagram showing an example of a browser image 400 displayed on the screen of a user terminal 20 that has received the analysis result data 173. Browser image 400 is an example of display image data generated by the generation unit 18. At the top of browser image 400, the words "<Bone Density Analysis>" are displayed to indicate the content of the image analysis. At the top left of browser image 400, the words "<Analysis Results>" are displayed to indicate that this is a screen displaying the analysis results. Below that, patient information may be displayed. Below that, the words "Your bone density is □ g / cm2" are displayed. Box 401 displays the estimated bone density value. Further below that, the words "Compared to young people, it is □%" may be displayed. Box 402 displays the ratio to the Young Adult Mean (YAM) bone density. Below that, the words "The probability of your femur fracture is □%" may be displayed. Box 406 displays the probability of fracture as a percentage. Further below that, the words "Judgment" and words such as "Bone loss" may be displayed in box 403. For example, if the ratio to YAM falls below 80%, it is judged as "bone loss," and if it falls below 70%, it is judged as a possibility of "osteoporosis." Browser image 400 may also have an "exit" button 404 and a "back" button 405. The "back" button may, for example, return to the previous screen, return to the home screen, or return to a specific screen.

[0043] (Generation part 18) Next, the details of the generation unit 18 will be described. In this embodiment, the generation unit 18 generates the derivation basis data 311 of the analysis results analyzed by the analysis unit 13. The communication control unit 11 may transmit the derivation basis data 311 generated by the generation unit 18 to the user terminal 20.

[0044] The analysis result data 173 is based on the output from the machine model 131, but the output from the machine model 131 does not include the analysis process. Therefore, it is generally not possible to judge the reliability of the output by looking only at the output from the machine model 131. By sending the analysis result data 173, including the derivation basis data 311, to the user terminal 20, it is possible to help improve the user's confidence in the analysis results.

[0045] For example, the derived basis data 311 may be processed image data 310 (basis image data) to which new information has been added to the image data 301 input to the machine model 131. For example, the new information may be information indicating the region in the image data 301 that was the main basis for deriving the analysis result. The processed image may be an image to which information indicating the main basis region has been added to the input image data. Information indicating a region is information that shows the extent of the region, such as coloring or a frame. In image analysis, a certain region of the image is often the main basis for estimation, so such information indicating a region allows the user to confirm the region that was the basis for estimation. In this case, the display image shown on the user terminal 20 includes a processed image to which the area of ​​interest information has been added to the medical image input to the machine model 131. The generation unit 18 may generate a display image in which the medical image (image data 301) and the processed image are displayed side by side. Such a display image makes it easy for the user to compare the original analyzed image and the processed image. Alternatively, the control unit 16 may switch between displaying the medical image and the processed image by user operation. For example, the control unit 16 may alternately display medical images and processed images when the user clicks a switching button.

[0046] The processed image data 310 does not need to contain all the information of the user-input image data 301. For example, the processed image data 310 may be a cropped version of the input image data 301, or it may be an image with a lower resolution than the input image data 301. This reduces the amount of data transmitted and received.

[0047] Figure 8 is a schematic diagram showing an example of a browser image 500 that transmits the derivation basis data 311 in addition to the analysis result data 173 to the user terminal 20. The browser image 500 is an example of display image data generated by the generation unit 18. The browser image 500 has the derivation basis data 311 (processed image data 502 including region 503) added to the analysis result data 173 shown in Figure 7.

[0048] Specifically, the browser image 500 displays the analysis result 501 along with the processed image data 502, which is the input image data (X-ray image data 300 in Figure 6) with a rectangular region 503 added. As shown in Figure 8, in this embodiment, the analysis result 501 derived by the machine model 131 that analyzes the image data, and the processed image data 502 including the derivation basis data 311 are displayed on a single screen. The browser image 500 may also display a "Back" button 506, an "Exit" button 507, and a "Future Prediction" button 505. The role of the "Future Prediction" button 505 will be described later.

[0049] Figure 9 is a magnified schematic diagram of the image of region 503 in Figure 8. Region 503 includes the four lumbar vertebrae shown from L1 to L4. This indicates that lumbar vertebrae L1-L4 are the region on which the analysis results were based. In fact, it is known that the bone density of lumbar vertebrae L1-L4 is related to the average bone density of the whole body. That is, the enclosed region 503 indicates that the analysis results of the mechanical model 131 were derived based on the bone density of these lumbar vertebrae L1-L4.

[0050] The region of focus on which the analysis results are based may include the segmented region of the medical image. The region of focus on which the analysis results are based may include at least a part of the segmented region of the medical image. The region of focus on which the analysis results are based may include all of the segmented region of the medical image. Segmentation is the process of dividing an image into several regions. Segmentation is performed to reduce the processing load of the machine model 131. That is, the machine model 131 may analyze only the segmented region. The segmented region can be set to any range. The segmented region may be, for example, a rectangle, a square, or a circle. If the segmented region is a square, the processing load of the machine model 131 can be reduced. When analyzing an X-ray image of the lumbar spine, for example, the range including lumbar vertebrae L1 to L4 is segmented. Therefore, the size of the segmented region may vary depending on the size of lumbar vertebrae L1 to L4 in the image. The segmentation region may be set to be slightly larger than the lumbar vertebrae L1-L4 in the direction of alignment of the lumbar vertebrae L1-L4, for example, or it may be set to overlap with the edges of the lumbar vertebrae L1-L4. The segmentation region may always have a predetermined dimension, or its dimensions may be set according to the medical image. For example, the segmentation region may identify the positions of the lumbar vertebrae L1-L4, set the length in the direction of alignment of the lumbar vertebrae L1-L4, and then set the length in the vertical direction of the lumbar vertebrae L1-L4. Segmentation may be performed by a machine model 131. The machine model 131 may have learned images with annotations of the analysis region in order to perform segmentation.

[0051] Furthermore, the generation unit 18 may generate a heatmap of the region that formed the basis of the analysis results. In this case, for example, the outer edge of the heatmap would point to the segmentation region. A heatmap is a method of representing the magnitude of bone density with the intensity of any color. For example, the generation unit 18 may generate a heatmap that shows the degree of focus. It may also generate a heatmap that shows the numerical value of bone density. Furthermore, the generation unit 18 may generate a heatmap that shows the possibility (probability) of fracture. The processed image may be an image in which a heatmap showing the region of focus information is superimposed on a medical image. The image used for the heatmap may be a still image or a video. By showing it as a video, for example, by fading various heatmaps in order, it becomes easy to visually recognize the relationships between each heatmap. Furthermore, if the analysis result is a heatmap of bone density that includes areas other than the segmentation region, a portion of the segmentation region may be enclosed in a frame.

[0052] The generation unit 18 may obtain information about the region that formed the basis of the analysis results from the analysis unit 13. Specifically, the generation unit 18 may obtain the region that formed the basis of the analysis results from the analysis unit 13 and generate information indicating that region (such as a border surrounding region 503). The generation unit 18 may obtain data indicating the degree of focus, bone density data, or information indicating the possibility of fracture within the region from the analysis unit 13 and generate a heat map. The generation unit 18 may generate any multiple of the following: a heat map showing the degree of focus data, a heat map showing bone density data, and a heat map showing the possibility of fracture. When displaying the heat map showing the degree of focus data, the heat map showing bone density data, and the heat map showing the possibility of fracture overlaid, the generation unit 18 may make the colors of the heat map showing the degree of focus data, the heat map showing bone density data, and the heat map showing the possibility of fracture different.

[0053] The machine model 131 analyzes, for example, X-ray image data 300 using a neural network model (NNM). In the NNM, the image is divided into small regions, each is quantified, and multiple regions are pooled to integrate them into larger regions, which are then quantified again. This process is repeated. Therefore, the machine model 131 may extract regions that have numerical values ​​that influence the processing results (for example, relatively large numerical values) as the basis regions.

[0054] In the example shown in Figure 8, a processed image data 502 was described in which the region 503 that formed the basis of the analysis result was superimposed on the X-ray image data 300 that was to be analyzed. However, the processed image data is not limited to this. For example, the analysis device 60A may transmit location information (such as coordinates) of the region that formed the basis of the analysis result in the image to be analyzed to the user terminal 20, and the user terminal 20 may display the processed image data with the region superimposed on the image to be analyzed.

[0055] A screen like the one shown in Figure 8 can be used when a doctor explains the analysis results to a patient. In other words, it can not only explain the analysis results to the patient, but also explain which areas of the image data were used as the basis for these results. This not only improves the doctor's confidence in the analysis results, but also makes it easier for the patient to accept the results.

[0056] In the example shown in Figure 8, the analysis results 501 and the processed image data 502, which includes the derivation basis data 311, are displayed on a single screen. However, "a single screen" does not mean that they are displayed on the screen simultaneously. The analysis results 501 and the processed image data 502 may be displayed by scrolling the screen up and down or left and right, for example. In other words, the range in which the analysis results 501 and the processed image data 502 are displayed by scrolling the screen up and down or left and right is referred to as "a single screen".

[0057] Figure 10 is a schematic diagram showing an example of a browser image 700 that displays the location of a bone fracture in a patient that is estimated from the image data analyzed by the analysis unit 13, the probability of that location fracturing, and the probability of that location fracturing in the patient three years from now. The future time is not limited to three years from now, but can be any time (for example, X years from now). Browser image 700 is an example of display image data generated by the generation unit 18. This is displayed by clicking the "Future Prediction" button 505 in the lower right corner of the browser image 500 shown in Figure 8. The prediction is not limited to the future, but can also be for a time different from the time the medical image was taken. Specifically, it can be a time before the time the medical image was taken, in which case "Past Prediction" can be displayed instead of or in addition to "Future Prediction".

[0058] Browser image 700 displays the text "<Bone Density Analysis>" at the top, indicating the subject of the image analysis. In the upper left of browser image 700, the text "<Future Prediction>" is displayed, indicating that this is a screen showing future predictions. Below that, patient information may be displayed. Below that, the text "Your femoral fracture probability is □%" is displayed. Box 701 displays the currently estimated femoral fracture probability. Further below, the text "Your femoral fracture probability in 3 years is □%" may be displayed. Box 702 displays the predicted fracture probability in 3 years. In addition, information regarding the basis for estimation or prediction may be displayed in conjunction with this image.

[0059] Figure 11 shows an example of a browser image 800 in which the derivation basis data has been added to the display image shown in Figure 10. Browser image 800 is an example of display image data generated by the generation unit 18. As shown in Figure 11, the browser image 800 displays an image 802 showing the derivation basis data in addition to the region 801 that shows the femoral fracture probability at the present time and 3 years from now, which was derived by the machine model 131.

[0060] Such browser images 800 can be used by physicians to explain the current and future risk of fractures to patients. Displaying information on the underlying rationale or predictive basis alongside the image can increase its persuasiveness to the patient.

[0061] (Machine Model 131) Next, the machine model 131 will be described. The machine model 131 is a model that estimates the bone condition of a subject. The input image data is an image containing bone, and the analysis result outputs an estimated result regarding the bone condition. For example, the machine model 131 is a trained model that has been trained to output estimated or calculated results regarding the bone condition, such as bone density, relative comparison of bone densities, and the probability of fracture, from X-ray images of bone. The bone density may be the calculated bone density of the bone region included in the image data, or it may be the average bone density of the whole body estimated from the image data. A known method can be used to calculate the bone density from the image. The relative comparison of bone densities is the ratio of the estimated bone density to YAM. The probability of fracture is the possibility of a fracture in a specific area (e.g., the femoral neck). The estimated result regarding the bone condition may be the estimated result at the time the image was taken, or it may be a prediction at a predetermined period of time after that time.

[0062] The machine model 131 may output at least one of the following as estimation results: bone density estimated at the time image data is acquired, bone density predicted after a predetermined period has elapsed from the time image data is acquired, fracture site and its probability estimated at the time image data is acquired, and fracture site and its probability predicted after a predetermined period has elapsed from the time image data is acquired.

[0063] (Learning method for machine model 131) Next, the learning method for the machine model 131 will be described. For example, learning to estimate bone density may be performed using X-ray images of bones with identified bone density as training data. Learning to estimate the likelihood of fracture may be performed using X-ray images of bones and data on whether the patient subsequently suffered a fracture within a predetermined period as training data. Relative comparison of bone density does not need to be learned; it can be obtained by dividing the estimated bone density by YAM. Furthermore, learning for future prediction may be performed using X-ray images of bones with identified bone density and data on what the patient's bone density was after a predetermined period, or whether or not a fracture occurred, as training data. By including lifestyle habits such as exercise, diet, smoking, and alcohol consumption of the patients used as training data, it is possible to construct a machine model 131 that can estimate or predict with even greater accuracy.

[0064] In Embodiment 2, a machine model 131 trained to estimate bone conditions such as bone density was used as an example. In this case, the input image data is an X-ray image of bone, and the output is an estimation result regarding the bone condition. However, the machine model 131 is not limited to this type of model. For example, the machine model 131 may be a cell pathology analysis model. In this case, the input medical image may be a microscopic image of the subject's cells, and the analysis result may be information regarding pathological mutations in those cells.

[0065] Figure 12 is a schematic diagram showing an example of a browser image 900, which is a display image with the cell pathology analysis results and the data for which they were derived added, when the machine model 131 is a cell pathology analysis model. Browser image 900 is an example of display image data generated by the generation unit 18. In the browser image 900 shown in Figure 12, the words "<Cell Pathology Analysis>" are displayed at the top to indicate the content of the image analysis. In the upper left of browser image 900, the words "<Judgment Result>" are displayed to indicate that this is a screen that displays the result of determining whether the cells were benign or malignant. Below that, patient information may be displayed. In the area 901 below that, the words "The probability of malignancy is □%" are displayed. The box displays a numerical value indicating the probability that the cells are malignant. Furthermore, to the right of area 901, image 902 is displayed with area 903 added as the data for which the analysis results were derived, which was the main basis for deriving the judgment result from the analyzed image. Area 903 is the area containing the cells that the machine model 131 determined to be malignant. Additionally, the browser image 900 may also display an "Exit" button 904 and a "Back" button 905.

[0066] Furthermore, a single screen may refer to a screen that is displayed by scrolling. Figure 13 is a schematic diagram showing an example of displaying images on a single screen by scrolling. The solid line region 1301 in Figure 13 is the display area of ​​the user terminal 20 (display device). The dotted line region 1302 is a single screen. The upper part of region 1302 displays the bone density analysis result 501 data, as explained in Figure 8. The lower part of region 1302 displays the processed image data 502, which is the basis derivation data, as explained in Figure 8. When region 1302 is first displayed, the bone density analysis result 501 data is displayed in region 1301, as shown in the upper part of Figure 13. However, when the user scrolls region 1302 upwards, the processed image data 502 appears, as shown in the upper part of Figure 13. Even with this configuration, the user can display the analysis results and the basis data for their derivation simply by scrolling the screen.

[0067] According to the configuration of the image remote analysis system 70A in the above embodiment 2, the derived basis data 311 can be provided to the user together with the analysis result data 173. This has the effect of improving the user's confidence in the analysis results. In addition, it has the effect of making it easier for patients to accept the analysis results when explaining them to them.

[0068] Next, the flow of the image remote analysis method S2 executed by the control unit 16 according to Embodiment 2 will be described. The image remote analysis method S2 includes steps S21 to S24 (not shown).

[0069] When a user accesses the analysis device 60A, the communication control unit 11 displays a browser image 200 on a web page via the communication unit 15, which is an input screen for inputting acquired information 172, including image data 301 to be analyzed, patient identification information 320, and attribute information 303 (S21).

[0070] The acquisition unit 12 acquires encrypted patient information 302, which is encrypted image data 301, attribute information 303, and patient identification information 320 contained in the acquired information 172 that is input into the browser image 200 and transmitted from the user terminal 20 (S22).

[0071] The analysis unit 13 analyzes the image data 301 sent from the acquisition unit 12 using the machine model 131 (S23).

[0072] The communication control unit 11 transmits the analysis result data 173 analyzed by the analysis unit 13, the derived basis data 311, and the encrypted patient information 302 to the user terminal 20 (S24).

[0073] According to the image remote analysis method S2 described above, the derived basis data 311 can be provided to the user together with the analysis result data 173. This has the effect of improving the user's confidence in the analysis results. In addition, it has the effect of making it easier for patients to accept the analysis results when explaining them to them.

[0074] [Embodiment 3] Other embodiments of this disclosure are described below. Embodiment 2 describes an analysis system that analyzes X-ray images of a patient's bones and outputs bone density, relative comparison of bone densities, and the probability of fracture (%). However, in images showing at least some bone (e.g., plain X-ray images), changes in the image may appear not only due to a decrease in bone density, but also due to other reasons. Other reasons include some disease, treatment marks, intentionally placed information, and items worn by the patient. Some disease may be, for example, arterial calcification, bone hardening, fracture, or organ tumors. Treatment marks may be, for example, implants or bone cement. Intentionally placed information may be, for example, letters such as "L" or "R" indicating direction. Items worn by the patient may be, for example, necklaces.

[0075] Due to these causes, the brightness (whiteness) of the image often changes compared to when there are no other reasons. Also, implants and necklaces may exhibit unique shapes. Such changes affect the bone density evaluation based on the analysis by the analysis unit 13 of the analysis device 60A of the remote image analysis system 70A described in Embodiment 2. Normally, a physician who views the image will notice such changes and interpret the bone density evaluation result taking into account the information that caused such changes. However, in some cases, a physician may not notice such changes in the image. Or, a physician may not be able to determine to what extent the analysis results of the analysis unit 13 take into account image changes due to other reasons. In such cases, a physician may be unsure how to interpret the analysis results of the analysis unit 13.

[0076] Therefore, it is desirable that the machine model be able to analyze not only the object itself but also changes in the image due to other reasons. The image remote analysis system 70B according to this embodiment can perform at least one of two analysis processes: the machine model analyzes the state of the object itself in the image (e.g., bone), and also detects changes in the image due to other reasons, i.e., reasons other than the state of the bone, and analyzes including those changes, or analyzes excluding those changes. Figure 14 is a block diagram showing the configuration of the image remote analysis system 70B according to Embodiment 3. Similar to the image remote analysis system 70A according to Embodiment 2, the image remote analysis system 70B includes an analysis device 60B (image remote analysis device) that analyzes medical data (e.g., medical image data) and a user terminal 20, and the analysis device 60B and the user terminal 20 are connected to communicate via the Internet.

[0077] The analysis device 60B comprises a communication unit 15, a control unit 16B, and a storage unit 17. The communication unit 15 and the storage unit 17 have the same functions as the communication unit 15 and storage unit 17 described in Embodiment 2, so their description is omitted here. The control unit 16B comprises a communication control unit 11, an acquisition unit 12, an analysis unit 13B, a generation unit (image generation unit) 18, and a determination unit 19. Of the control unit 16B, all units except the analysis unit 13B have the same functions as the units described in Embodiment 2, so their description is omitted here.

[0078] The analysis unit 13B comprises a mechanical model 131 and a second mechanical model 132. Mechanical model 131 is a mechanical model that performs analysis on the state of an object (e.g., bone), similar to Embodiment 2. In contrast, the second mechanical model 132 is a mechanical model that has been trained to detect image changes due to other reasons, such as some disease, treatment marks, various intentionally added information, or items worn by the patient. The second mechanical model 132 can learn the position of images due to some disease, treatment marks, various intentionally added information, or items worn by the patient using images annotated by a physician. Mechanical model 131 can perform analysis on the state of an object, including image changes due to other reasons detected by the second mechanical model 132. Furthermore, mechanical model 131 can perform analysis on the state of an object, excluding image changes due to other reasons detected by the second mechanical model 132.

[0079] The analysis unit 13B includes a machine model, which performs an analysis of the state of the target object (e.g., bone) itself, and may also detect changes in the image due to other reasons and perform two analysis processes: one that includes the changes in the analysis, and another that excludes the changes in the analysis.

[0080] Figure 15 is an example of an image generated by the generation unit 18 based on the analysis results of the second machine model 132, which includes data for the derivation of image changes due to other reasons. The area indicated by frame 1502 in Figure 15 shows the area where the spinal implant is visible. The area indicated by frame 1503 shows the area where the calcified abdominal aorta is visible. In the figure, each area is shown with a dotted line, but they may also be shown with different colors, such as a red frame and a yellow frame. The area indicated by frame 1501, similar to Embodiment 2, shows the area that serves as the basis for the derivation of the bone density analysis results of the lumbar spine of the machine model 131. Thus, the range for detecting image changes due to other reasons may be outside the area indicated by frame 1501, which serves as the derivation basis when an analysis of the state of the object is performed.

[0081] The analysis unit 13B may perform an analysis of the bone condition, including changes in the image due to other reasons. Alternatively, the analysis unit 13B may perform an analysis of the bone condition, excluding changes in the image due to other reasons. The generation unit 18 may also generate analysis results for both cases, including and excluding changes in the image due to other reasons. The user may specify which analysis method to use and which analysis results to display. If there are multiple areas of change, the system may be configured so that the user can specify which analysis method to use and which analysis results to display for each of them.

[0082] Figure 16 is an example of a browser image 1600 that displays the analysis results in both cases: with and without image changes due to other reasons. This browser image 1600 displays, for example, the analysis image 1602 and the analysis result 1601. The analysis image 1602 is the same as the image shown in Figure 15. The analysis result 1601 displays "If the dotted line area is included, your bone density is XX g / cm2" and "If the dotted line area is excluded, your bone density is YY g / cm2". At the bottom of the analysis image 1602 is a selection button 1606, which, when clicked, allows you to move to a screen where you can select whether to include one or both of the dotted line areas 1604 and 1605 in the analysis, and which analysis result to display. The back button 1607 and the exit button 1608 remain the same as before.

[0083] The above describes an example in which the image remote analysis system 70B evaluates bone density of the femur or vertebral bone. However, the objects to be evaluated are not limited to these. For example, the image remote analysis system 70B may evaluate X-ray images of bones in the chest, or it may analyze images of objects other than bone.

[0084] As described above, according to the image remote analysis system 70B of this embodiment, as shown in Figure 15, the region indicated by frame 1501, which is the basis for deriving analysis results regarding the state of bone, and the regions indicated by frames 1502 and 1503, which are the basis for deriving image changes due to other reasons, can be displayed within a single image. Furthermore, as shown in Figure 16, it is possible to display whether the analysis results include image changes due to other reasons or whether the analysis results exclude image changes due to other reasons. With this configuration, a physician can confirm the conditions under which the analysis results of the image output by the image remote analysis system 70B were obtained. Therefore, the interpretation of the analysis results of the image output by the image remote analysis system 70B can be accurately performed.

[0085] [Examples of implementation using software] The functions of the image remote analysis system 70A (hereinafter referred to as "the system") are programs that cause a computer to function as the system, and these functions can be realized by programs that cause a computer to function as each part of the system.

[0086] In this case, the system includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., memory) as hardware for executing the program. By executing the program using this control device and storage device, the functions described in each of the embodiments are realized.

[0087] The above program may be recorded on one or more computer-readable recording media, not temporary ones. These recording media may or may not be provided by the above device. In the latter case, the program may be supplied to the above device via any wired or wireless transmission medium.

[0088] Furthermore, some or all of the functions of the above-mentioned parts can also be realized by logic circuits. For example, an integrated circuit in which logic circuits functioning as the above-mentioned parts are formed is also included in the scope of this disclosure. In addition, it is also possible to realize the functions of the above-mentioned parts by, for example, a quantum computer.

[0089] The inventions described in this disclosure have been explained above based on the drawings and embodiments. However, the inventions described in this disclosure are not limited to the embodiments described above. That is, the inventions described in this disclosure can be modified in various ways within the scope shown in this disclosure, and embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the inventions described in this disclosure. In other words, it should be noted that it is easy for those skilled in the art to make various modifications or alterations based on this disclosure. Furthermore, it should be noted that these modifications or alterations are included in the scope of this disclosure.

[0090] This disclosure describes, but is not limited to, the analysis of medical images and the modification of medical images based on focus region information related to the focus region identified during the process of outputting the analysis results. For example, analysis may be performed using a pre-trained machine model with only numerical data or the information shown below as input information, without including medical images. In this case, the focus element information identified during the process of outputting the analysis results may be displayed on the displayed image along with the analysis results. Furthermore, the focus element information may be a part of the input information that has been highlighted. For example, if age and gender information are focus elements, the age and gender items or numerical values ​​may be highlighted in a conspicuous color. Medical images may, for example, be images taken of a subject using an endoscope. More specifically, medical images include images taken with an endoscope of a part of the subject that includes at least one of the following: the nasal cavity, esophagus, stomach, duodenum, rectum, large intestine, small intestine, anus, and colon. Medical images of these parts may output analysis results that highlight, for example, a site of interest that includes at least one of inflammation, polyps, and cancer, using a learning model. In such a case, the learning model can be, for example, a model trained on a first training image containing an image of the site of interest and first training data indicating the presence of the site of interest, and a second training image containing an image without the site of interest and second training data indicating the absence of the site of interest. The first training data may include information indicating the degree of inflammation (degree of inflammation) or malignancy (degree of malignancy) of the site of interest. The analysis results may, for example, be a display that surrounds the site of interest, a display that points to the site of interest, or a display that superimposes color on the site of interest. Along with these analysis results, the derivation data showing the basis for the derived analysis results may be displayed in the same manner as in the embodiment described above. Alternatively, the medical images may be, for example, images of the subject's eyes, skin, etc., taken with a digital camera. The medical images of these areas may output analysis results that, for example, highlight the signs of interest using a learning model. The signs of interest may include, for example, signs indicating diseases such as glaucoma, cataracts, age-related macular degeneration, conjunctivitis, stye, retinopathy, and blepharitis if they are of the eyes. Alternatively, the signs of interest may include signs such as skin cancer, urticaria, atopic dermatitis, and herpes if they are of the skin. The analysis results may be a display that surrounds these signs of interest, a display that points to the signs of interest, a display that superimposes color on the signs of interest, or a display that shows the name of the disease. As a learning model, for example, a model learned based on a first training image having images of these areas of interest and first training data showing the presence of the signs of interest, and a second training image having images without signs of interest and second training data showing the absence of signs of interest can be used. Along with these analysis results, the derivation data showing the basis for the derived analysis results may be displayed in the same manner as in the embodiment described above.

[0091] The input information may include age, sex, weight, height, presence or absence of fractures, location of fractures, fracture history, family history of fractures (e.g., parents), underlying medical conditions (e.g., food and / or drug allergies, diseases related to and unrelated to the onset of osteoporosis, etc.), smoking history, drinking habits (e.g., frequency and amount of alcohol consumed, etc.), occupational history, exercise history, medical history (e.g., history of bone disease), menstruation (e.g., cycle and presence or absence, etc.), menopause (e.g., possibility and presence or absence, etc.), artificial joints (e.g., type, presence or absence, and timing of replacement surgery for spinal implants or knee joints, etc.), blood test results, urine test results, medications being taken, and gene sequences.

[0092] (summary) (Aspect 1) An image generation apparatus according to Embodiment 1 of the present disclosure comprises: an acquisition unit that acquires analysis results output from a machine model that analyzes medical images of a subject, and focus region information relating to a region within the medical image that was focused on during the process of outputting the analysis results; and an image generation unit that generates a display image by modifying the medical image based on the focus region information.

[0093] (Aspect 2) In the image generation apparatus according to aspect 2 of the present disclosure, in aspect 1 above, the displayed image includes a processed image to which the area of ​​interest information has been added to the medical image input to the machine model.

[0094] (Aspect 3) In the image generation apparatus according to embodiment 3 of the present disclosure, in embodiment 1 or 2 above, the region of interest includes the region in which the medical image is segmented.

[0095] (Aspect 4) In the image generation apparatus according to embodiment 4 of this disclosure, the area of ​​interest information is shown as a heat map in any one of embodiments 1 to 3 above.

[0096] (Appendix 5) In the image generation apparatus according to Embodiment 5 of this disclosure, in Embodiment 2 above, the processed image is an image on which the area of ​​interest information is superimposed.

[0097] (Aspect 6) In the image generation apparatus according to embodiment 6 of the present disclosure, in embodiment 2 or 5 above, the image generation unit generates the display image in which the medical image and the processed image are arranged in parallel.

[0098] (Aspect 7) In the image generation apparatus according to embodiment 7 of this disclosure, in any one of embodiments 2, 5, or 6 above, the display image and the processed image can be swapped by operation.

[0099] (Pattern 8) The image generating apparatus according to aspect 8 of the present disclosure is, in any one of aspects 1 to 7 above, wherein the machine model is a model for estimating the bone condition of the subject, the medical image is an image showing the bones of the subject, and the analysis result includes an estimation result regarding the bone condition of the subject.

[0100] (Aspect 9) In the image generation apparatus according to embodiment 9 of the present disclosure, in any one of embodiments 1 to 8, the displayed image is shown as a heatmap in which the analysis result and the area of ​​interest information are each in different embodiments.

[0101] (Aspect 10) In the image generation apparatus according to aspect 10 of the present disclosure, in aspect 8, the estimation result is at least one of the following: the estimated location of the fracture of the subject, and the probability that the subject will suffer a fracture.

[0102] (Aspect 11) The image generating apparatus according to embodiment 11 of the present disclosure is, in any one of embodiments 1 to 7 above, wherein the machine model is a cell pathology analysis model, the medical image is a microscopic image of the subject's cells, and the analysis result includes information on pathological mutations in the subject's cells.

[0103] (Aspect 12) The image generating apparatus according to aspect 12 of this disclosure is such that, in any one of aspects 1 to 11 above, the display image is a display image that is displayed on a single screen.

[0104] (Aspect 13) In the image generation apparatus according to aspect 13 of the present disclosure, in aspect 1, the machine model detects changes in the image due to reasons other than the state of the object and performs at least one of two analysis processes: analyzing the image including the changes, and analyzing the image excluding the changes.

[0105] (Aspect 14) The image generating apparatus according to aspect 14 of the present disclosure, in aspect 13, includes a mechanical model that makes estimations about the state of the object itself, and a second mechanical model that detects changes in the image for reasons other than the state of the object.

[0106] (Aspect 15) The image generating device according to aspect 15 of this disclosure is capable of displaying at least one of the results of the analysis including the changes in the image, and the results of the analysis excluding the changes, in aspect 13 or 14.

[0107] (Aspect 16) An image generation method according to aspect 16 of this disclosure includes an acquisition step of acquiring analysis results output from a machine model that analyzes medical images of a subject, and focus region information relating to a region within the medical image that was focused on during the process of outputting the analysis results, and an image generation step of generating a display image by modifying the medical image based on the focus region information.

[0108] (Aspect 17) The program relating to aspect 17 of this disclosure is an image generation program for causing a computer to function as an image generation device as described in aspect 1, and is an image generation program for causing a computer to function as the acquisition unit and the image generation unit.

[0109] (Aspect 18) The recording medium according to aspect 18 of this disclosure is a computer-readable non-temporary recording medium on which the image generation program described in aspect 17 is recorded.

[0110] (Aspect 19) The display device according to aspect 19 of this disclosure displays analysis results output from a machine model that analyzes medical images of a subject, and a processed image obtained by processing the medical image based on focus area information relating to a region within the medical image that was identified during the process of outputting the analysis results. [Explanation of symbols]

[0111] 70A, 70B... Image Remote Analysis System 3. Image generation device 11,33...Communication Control Unit 12...Acquisition part 13...Analysis department 131... Mechanical Model 14. Analysis Result Transmission Section 15,50... Communications Department 16,30...Control Unit 17,40...Storage section 18...Generation section 19... Judgment section 20. User terminals 31. Image generation unit 32. Data acquisition unit (acquisition unit) 41. Analysis data 42. Derivation basis data 43...Displayed image data 60,60A,60B...Analysis equipment 601... Mechanical Model 70...Display device

Claims

1. An acquisition unit that acquires derivation basis data indicating the information used as the basis for the analysis results in a machine model that analyzes input information including X-ray images showing the bones of the subject, An image generation system comprising: an image generation unit that generates a processed image obtained by modifying the X-ray image based on the aforementioned derived data.

2. The image generation system according to claim 1, wherein the processed image includes information indicating a region based on the derived data.

3. The image generation system according to claim 2, wherein the information indicating the region includes at least one of a frame, coloring, and a heat map.

4. The image generation system according to claim 1, wherein the processed image is a cropped image of the X-ray image, or an image with a lower resolution than the X-ray image.

5. The image generation system according to claim 1, wherein the derived data is information relating to a region of interest that includes at least a portion of the segmentation region of the X-ray image.

6. The image generation system according to claim 5, wherein the shape of the segmentation region is rectangular.

7. The image generation system according to claim 1, wherein the image generation unit further generates a display image in which the processed image and the X-ray image are positioned side by side.

8. The image generation system according to claim 1, wherein the X-ray image and the processed image are displayed interchangeably by operation.

9. The image generation system according to claim 1, wherein the analysis results include at least one of an estimated bone density, a ratio to the average bone density of young adults, and an estimated fracture site.

10. The image generation system according to claim 1, wherein the image generation unit further generates a display image including the processed image and characters for determination based on the analysis results.

11. The image generation system according to claim 10, wherein the processed image and the judgment text based on the analysis result are displayed interchangeably by operation.

12. The display image further includes the X-ray image, The image generation system according to claim 10, wherein the characters for the determination based on the analysis results and the X-ray image are displayed interchangeably by operation.

13. The analysis results include the possibility of fracture in the subject at the first time point, The image generation system according to claim 10, wherein the displayed image further includes a button that displays the possibility of fracture of the subject at a first time point in response to an operation.

14. The image generation system according to claim 13, wherein the analysis results further include the possibility of fracture at a second time point different from the first time point.

15. The image generation system according to claim 1, wherein the image generation unit further generates display images including heatmaps of different configurations that show the analysis results and the derivation basis data, respectively.

16. The image generation unit further generates a display image including at least one of the X-ray image and the characters for the determination based on the analysis results, and the processed image. The image generation system according to claim 1, wherein the displayed image comprises a first region displayed on a screen and a second region displayed on the screen by scrolling the screen displaying the first region.

17. The image generation unit further generates a display image that includes a first display area including the X-ray image, a second display area including characters for judgment based on the analysis results, and a third display area including the processed image. The image generation system according to claim 1, wherein the first display area, the second display area, and the third display area are interchangeable by operation.

18. The aforementioned machine model further comprises a second machine model that detects image changes due to reasons other than the bone condition of the subject, The image generation system according to claim 1, wherein the analysis results include at least one of the results of analysis including the changes in the image and the results of analysis excluding the changes in the image.

19. The process involves an acquisition step to obtain derivation basis data that shows the information used as the basis for the analysis results obtained by a machine model that analyzes input information including X-ray images showing the bones of the subject, and An image generation method comprising: an image generation step of generating a processed image by modifying the X-ray image based on the derived basis data.

20. An image generation program for causing a computer to function as an image generation system according to claim 1, wherein the computer functions as the acquisition unit and the image generation unit.

21. A computer-readable non-temporary recording medium that stores the image generation program described in claim 20.

22. A display system that displays a processed image generated by modifying an X-ray image based on derivation basis data, which indicates the information used as the basis in the process by which a machine model that analyzes input information including an X-ray image showing the bones of a subject derives its analysis results.

Images