Medical information processing device, medical information processing system, medical information processing program, and medical information processing method

Abstract

To enable healthcare professionals to easily understand that an image or a part of an image was generated by a generative model when performing measurements using that image. [Solution] The medical information processing device according to the embodiment comprises an image acquisition unit, a measurement unit, and a control unit. The image acquisition unit inputs medical image data to a generation model and acquires generated image data generated by the generation model. The measurement unit measures the measurement range using the generated image data. The warning unit outputs predetermined information if the generated region in the generated image data includes at least a part of the measurement range.

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to a medical information processing device, a medical information processing system, a medical information processing program, and a medical information processing method.

Background Art

[0002] In recent years, medical image data is generated by scanning a subject with a medical imaging device such as an X-ray CT (Computed Tomography) device. The medical image data may contain artifacts due to metals, beam hardening, etc. Thus, when the medical image data contains noise such as artifacts, the generation model can generate image data with reduced artifacts.

[0003] However, the image generated by the generation model is not the same as the original image obtained by scanning the subject. Therefore, it is desirable for medical staff to perform radiography and the like while grasping that the image is generated by the generation model.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to easily let medical staff grasp that the image or a part of the image is generated by a generation model when the medical staff performs measurement using the image. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problems. The problems corresponding to the respective effects of the respective configurations shown in the embodiments described later can also be regarded as other problems. [Means for solving the problem]

[0006] The medical information processing device according to the embodiment comprises an image acquisition unit, a measurement unit, and a control unit. The image acquisition unit inputs medical image data into a generation model and acquires generated image data generated by the generation model. The measurement unit uses the generated image data to measure the measurement range. The control unit outputs predetermined information if the generated region in the generated image data includes at least a part of the measurement range. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 shows an example of a medical information processing system according to the first embodiment. [Figure 2] Figure 2 shows an example of a processing flow using the model control function. [Figure 3] Figure 3 shows an example of a generated image in which the generated region can be identified. [Figure 4] Figure 4 shows an example of a measurement operation process performed by the medical information processing device according to the first embodiment. [Figure 5] Figure 5 shows an example of a generated image in which the generated region can be identified. [Figure 6] Figure 6 shows an example of a medical information processing system according to the second embodiment. [Figure 7] Figure 7 shows an example of a generated image in which the generated region can be identified. [Figure 8] Figure 8 shows an example of a medical information processing system according to the third embodiment. [Figure 9] Figure 9 shows an example of measurement using a generated image. [Figure 10] Figure 10 shows an example of measurement using a generated image. [Figure 11] Figure 11 shows an example of measurement using a generated image. [Figure 12]Figure 12 shows an example of measurement using a generated image. [Figure 13] Figure 13 shows an example of measurement using a generated image. [Figure 14] Figure 14 shows an example of measurement using a generated image. [Figure 15] Figure 15 shows an example of measurement using three-dimensional medical image data. [Modes for carrying out the invention]

[0008] Hereinafter, with reference to the drawings, a medical information processing device, a medical information processing system, a medical information processing program, and a medical information processing method according to this embodiment will be described. In the following embodiments, parts with the same reference numerals perform similar operations, and redundant explanations will be omitted as appropriate.

[0009] (First embodiment) Figure 1 shows an example of a medical information processing system 1 according to the first embodiment. As shown in Figure 1, the medical information processing system 1 includes a medical information management system 10, a generation model server 20, and a medical information processing device 30. The medical information processing system 1 is a system that includes a generation model server 20 that generates at least a portion of medical image data, and a medical information processing device 30 that displays images generated by a generation model 21 possessed by the generation model server 20. The medical information management system 10, the generation model server 20, and the medical information processing device 30 are connected to each other via a network 40 such as a LAN (Local Area Network). Note that the medical information processing system 1 shown in Figure 1 is just an example, and may have other devices or systems.

[0010] The medical information management system 10 is a system for storing and managing medical information. For example, the medical information management system 10 is implemented by one or more servers, personal computers, or other computer equipment. Medical information is, for example, medical image data, or information containing medical image data.

[0011] The medical information management system 10 may be, for example, an electronic medical record system, a hospital information system (HIS), a clinical laboratory information system (LIS), a radiology information system (RIS), or a picture archiving and communication system (PACS), etc. Further, the medical information management system 10 may include a plurality of the above systems and devices, etc.

[0012] The generation model server 20 includes a generation model 21. Further, the generation model server 20 has a memory circuit and a processor. Also, the generation model 21 is stored, for example, in the memory circuit of the generation model server 20 and operates by the processor. Note that the configuration of the generation model server 20 is not particularly limited, and a known configuration can be adopted.

[0013] When the generation model 21 receives a prompt, it generates generated image data corresponding to the prompt. The generation model 21 is, for example, a learned model such as a multimodal model (MMM) or a large language model (LLM) that can handle data of review types such as image data. The generation model 21 is an integrated learned model that can process a plurality of types of modalities (data types) such as text, images, voice, and numerical values at once. Also, the generation model 21 may use information such as a database owned by a user who uses the generation model 21, for example, like retrieval-augmented generation (RAG), to generate the generated image data.

[0014] The generation model 21 generates generated image data by generating at least a part of the input medical image data based on a prompt. For example, the generation model 21 generates an image with reduced noise such as artifacts included in the medical image data based on a prompt. Note that the generation model 21 is not limited to an image with reduced noise such as artifacts, and may generate information outside the imaging region in the medical image data, or may generate an image from which a device related to treatment in the medical image data has been removed. The generation model 21 generates information on a region outside the imaging region (imaging range or reconstruction range) as information outside the imaging region. For example, the generation model 21 generates an image corresponding to the head region for an image captured with the imaging range set from the neck to the waist. For example, the generation model 21 generates a region outside the image (for example, the region where the ribs and a part of the lungs are cut off when the enlarged reconstruction image of the heart is used). A device related to treatment is a device such as a coil, a stent, or a clip that is inserted into a blood vessel or that sandwiches a blood vessel. For example, the generation model 21 generates an image in which the device related to treatment is not depicted from an image in which the device related to treatment is depicted as generation of an image from which the device related to treatment has been removed. For example, the generation model 21 generates by estimating information without the device related to treatment from information around the device related to treatment.

[0015] Note that the generation model 21 is not limited to the generation model server 20, and may be possessed by other devices. For example, the generation model 21 may be possessed by the medical information processing device 30.

[0016] The medical information processing device 30 is a device operated by medical staff. For example, the medical information processing device 30 is realized by a computer device such as a personal computer.

[0017] The medical information processing device 30 includes a NW (network) interface 31, a storage circuit 32, an input interface 33, a display 34, and a processing circuit 35.

[0018] The NW interface 31 is connected to the processing circuit 35 and controls the transmission and communication of various data between the device connected via the network 40. For example, the NW interface 31 can be implemented by a network card, network adapter, NIC (Network Interface Controller), etc.

[0019] The memory circuit 32 is connected to the processing circuit 35 and stores various types of information in advance. The memory circuit 32 also stores various programs. The memory circuit 32 is, for example, a non-volatile storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or integrated circuit memory device that stores various types of information. In addition to HDDs and SSDs, the memory circuit 32 may also be a drive device that reads and writes various types of information to portable storage media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), flash memory, or semiconductor memory elements such as RAM (Random Access Memory). The memory circuit 32 is an example of a storage unit.

[0020] The input interface 33 is connected to the processing circuit 35 and converts input operations received from the operator (medical professional) into electrical signals and outputs them to the processing circuit 35. Specifically, the input interface 33 converts input operations received from the operator into electrical signals and outputs them to the processing circuit 35. For example, the input interface 33 can be implemented by a trackball, switch buttons, mouse, keyboard, touchpad that performs input operations by touching the operating surface, touchscreen that integrates a display screen and a touchpad, non-contact input circuit using an optical sensor, and audio input circuit. In this specification, the input interface 33 is not limited to those equipped with physical operating components such as a mouse or keyboard. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the device and outputs these electrical signals to a control circuit is also included as an example of the input interface 33.

[0021] The display 34 is connected to the processing circuit 35 and displays various information and image data output from the processing circuit 35. For example, the display 34 can be implemented as a liquid crystal display, a CRT (Cathode Ray Tube) display, an organic EL display, a plasma display, a touch panel, etc.

[0022] The processing circuit 35 controls the operation of the entire medical information processing device 30. The processing circuit 35 has, for example, a prompt acquisition function 351, a model control function 352, an output control function 353, and a measurement function 354. In this embodiment, each processing function performed by the constituent elements, the prompt acquisition function 351, the model control function 352, the output control function 353, and the measurement function 354, is stored in the memory circuit 32 in the form of a program that can be executed by a computer. The processing circuit 35 is a processor that realizes the functions corresponding to each program by reading the program from the memory circuit 32 and executing it. In other words, the processing circuit 35 in the state in which each program has been read will have the functions shown in the processing circuit 35 of Figure 1.

[0023] In Figure 1, the prompt acquisition function 351, model control function 352, output control function 353, and measurement function 354 are described as being implemented by a single processor. However, the processing circuit 35 may be configured by combining multiple independent processors, with each processor executing a program to implement the functions. Also, in Figure 1, a single memory circuit such as the memory circuit 32 is described as storing the program corresponding to each processing function. However, multiple memory circuits may be distributed and the processing circuit 35 may be configured to read the corresponding program from each individual memory circuit.

[0024] In the above description, the term "processor" refers to circuits such as a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), an Application Specific Integrated Circuit (ASIC), or a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). The processor functions by reading and executing a program stored in the memory circuit 32. Alternatively, instead of storing the program in the memory circuit 32, the processor may be configured to directly incorporate the program into its circuitry. In this case, the processor functions by reading and executing the program incorporated into the circuitry.

[0025] The prompt acquisition function 351 acquires prompts to be input to the generation model 21. More specifically, the prompt acquisition function 351 controls the input interface 33 to acquire prompts indicating the content of the instructions for the generation model 21.

[0026] A prompt is, for example, data indicating instructions for the generative model 21. For example, a prompt can be information such as text data or image data. A prompt includes instruction content information and instruction target information. The instruction content information indicates instructions for the generative model 21. For example, the instruction content information is text data instructing the reduction of artifacts contained in medical image data. The instruction content information may also include artifact location information indicating the location of the artifacts to be reduced within the medical image data. The artifact location information may identify the location of the artifacts by a part of the subject, such as a lung field region, by coordinates, or by other methods.

[0027] The information to be indicated is information that indicates the input image data to be indicated. For example, the input image data is medical image data that has regions to be processed by the generative model 21, such as artifacts. For example, the input image data is CT image data. However, the input image data is not limited to CT image data; it may also be X-ray image data, magnetic resonance image data, ultrasound image data, or other medical image data.

[0028] For example, the information to be indicated may be identification information to identify the input image data to be indicated, or a path indicating the location where the input image data is stored. Alternatively, the information to be indicated may be the input image data itself. In this case, the prompt acquisition function 351 acquires a prompt that includes medical image data.

[0029] The model control function 352 controls the generation model 21. More specifically, the model control function 352 inputs prompts obtained by the prompt acquisition function 351 to the generation model 21. The model control function 352 also acquires the generated image data output from the generation model 21. The model control function 352 is an example of an image acquisition unit. For example, the model control function 352 acquires generated image data with reduced artifacts contained in the input image data, using the generation model 21 which generates images with reduced artifacts.

[0030] Figure 2 shows an example of the processing flow by the model control function 352. Figure 2(a) shows an example of an input image G1 based on input image data. Figure 2(b) shows an example of an image generation region R1 on the input image G1 that is processed by the generation model 21. Figure 2(c) shows an example of a generated image G2 based on the generated image data output by the generation model 21 that received the input image data.

[0031] As shown in Figure 2(a), the input image G1 has an artifact region A1, which is an artifact region. As shown in Figure 2(b), the generation model 21 generates image data of the image generation region R1, which includes the artifact region A1. In other words, the image generation region R1 is the region from which the image is generated by the generation model 21. For example, the image generation region R1 may be a region containing artifacts, an area outside the imaging region in the input image data, or an area in the input image data that includes a treatment device. The generation model 21 then generates an image with reduced artifacts, an image from which information outside the imaging region has been generated, or an image from which treatment devices have been removed. Figure 2(b) shows an example where a rectangular region (ROI: Region of Interest) is set as the image generation region R1, but the shape of the image generation region R1 is not limited to this. For example, the shape of the image generation region R1 may be circular, a polygon other than a rectangle, or a shape that follows the contour of the artifact region A1. Furthermore, the image generation region R1 may be a two-dimensional region or a three-dimensional region. As shown in Figure 2(c), the generation model 21 outputs a generated image G2, which is an image generated from the image generation region R1. The model control function 352 acquires the generated image data generated by the generation model 21.

[0032] Furthermore, the model control function 352 may acquire image generation likelihood information if it is output from the generation model 21. The image generation likelihood information is information that includes information indicating the position of the generation region R2 (see Figure 3) corresponding to the image generation region R1, and likelihood information for each sub-region obtained by subdividing the generation region R2. Here, the image generation region R1 is the region in the input image data that will be imaged by the generation model 21. On the other hand, the generation region R2 is the region of the image generated by the generation model 21 in the generation image data. For example, the information indicating the position of the generation region R2 is the coordinates indicating the position of the generation region R2 in the generation image data. The likelihood information is the degree to which the image of each sub-region included in the generation region R2 generated by the generation model 21 is likely to be. In other words, the image generation likelihood information is information that associates the position of each sub-region of the generation region R2 with the likelihood of each sub-region. The sub-regions may be of any size or shape. The sub-regions are set by a predetermined method and their likelihoods are associated with them. For example, the generative model 21 defines subregions by dividing them into pre-set pixel counts. Then, the generative model 21 sets the likelihood corresponding to each subregion.

[0033] The output control function 353 outputs predetermined information if the generated region R2 in the generated image data includes at least a portion of the measurement range of the measurement function 354. For example, the output control function 353 displays or notifies predetermined information as output of predetermined information. For example, the output control function 353 displays the generated image G2 on the display 34 based on the generated image data. For example, the output control function 353 displays a generated image G2 that can identify the generated region R2, which is the area of ​​the image generated by the generation model 21.

[0034] Figure 3 shows an example of a generated image G2 that can identify the generated region R2. For example, the output control function 353 displays a likelihood notification image R3, which has the same shape as the generated region R2, on the generated region R2 based on the image generation likelihood information. The likelihood notification image R3 is an image that shows the shape of the generated region R2 and the location of the generated region R2. In addition, the likelihood notification image R3 is an image that notifies the likelihood of each sub-region of the generated region R2 in numerical terms. That is, the likelihood notification image R3 is an image that makes the generated region R2 identifiable. More specifically, the likelihood notification image R3 is an image that has lines dividing the generated region R2 into each sub-region and numerical values ​​indicating the likelihood of each sub-region. Note that the output control function 353 does not display the generated region R2, but displays the likelihood notification image R3 as an image that represents the generated region R2. The output control function 353 displays the likelihood (numerical value) for each sub-region of the generation region R2 for the likelihood notification image R3, but it may also display the likelihood (numerical value) for each pixel of the generation region R2.

[0035] As shown in Figure 3, the output control function 353 displays a likelihood notification image R3 for each of the multiple subregions included in the generation region R2, indicating the likelihood of the image generated by the generation model 21. For example, the output control function 353 displays a generated image G2 in which the likelihood notification image R3 is superimposed on the generation region R2. This allows the output control function 353 to identify the location where the generation region R2, which is the region of the image generated by the generation model 21, exists. Therefore, healthcare professionals can grasp the presence of the generation region R2 by using the likelihood notification image R3.

[0036] The measurement function 354 measures the measurement range using the generated image G2 based on the operation received via the input interface 33. The measurement function 354 is an example of a measurement unit. More specifically, the measurement function 354 accepts the specification of the measurement range in the generated image G2 based on the operation received via the input interface 33. Then, the measurement function 354 measures the measurement range included in the generated image data.

[0037] More specifically, the measurement function 354 measures the specified measurement range in the generated image G2. For example, the measurement function 354 measures the size of each part of the subject included in the generated image G2, as well as the size of the shadows. Specifically, the measurement function 354 measures distance, area, and volume. This allows the measurement function 354 to measure the volume of the extracted tumor region, or to measure the cross-sectional area and cross-sectional area ratio of the brain in cases of brain atrophy.

[0038] Here, the output control function 353 outputs predetermined information if the generated region R2 in the generated image data includes at least a portion of the measurement range of the measurement function 354. The output control function 353 is an example of a control unit. For example, the output control function 353 warns that the generated region R2 is being measured if the generated region R2 in the generated image data generated by the generation model 21, from which artifacts have been reduced by the generation model 21, includes the measurement range measured by the measurement function 354. This allows medical professionals to measure the measurement range while being aware that the image was generated by the generation model 21. The measurement function 354 may measure the measurement range using the generated image G2 based on other specifications, not just operations received through the input interface 33.

[0039] The output control function 353 may notify the measurement function 354 to perform a remeasurement if the generated region R2 in the generated image data includes the measurement range measured by the measurement function 354. For example, the output control function 353 may notify the measurement function to remeasure the measurement range excluding the generated region R2. The measurement function 354 may then remeasure the measurement range excluding the generated region R2.

[0040] Next, the measurement operation process performed by the medical information processing device 30 will be described. The measurement operation process is the process of measuring a specified measurement range using the generated image data generated by the generation model 21.

[0041] Figure 4 shows an example of a measurement operation process performed by the medical information processing device 30 according to the first embodiment.

[0042] The prompt acquisition function 351 acquires the prompt to be input to the generation model 21 (step S1).

[0043] The model control function 352 inputs the prompt obtained by the prompt acquisition function 351 to the generation model 21 (step S2).

[0044] The model control function 352 obtains image generation data from the generation model 21 in which an image of the generation region R2 corresponding to the image generation region R1 has been generated (step S3). The model control function 352 may also obtain image generation data from the generation model 21 that shows the likelihood of each sub-region obtained by subdividing the generation region R2.

[0045] The model control function 352 obtains image generation likelihood information from the generation model 21, which shows the likelihood of each sub-region obtained by subdividing the generation region R2 (step S4). Note that if the generation region R2 is known, step S4 is not required.

[0046] The output control function 353 displays the generated image G2 based on the generated image data generated by the generation model 21 (step S5).

[0047] The measurement function 354 accepts the specification of the measurement range based on the operation received by the input interface 33 (step S6).

[0048] The output control function 353 determines whether the specified measurement range is included in the generation region R2 (step S7). If the specified measurement range is included in the generation region R2 (step S7; Yes), the output control function 353 displays a warning indicating that the measurement range is included in the generation region R2 (step S8).

[0049] If the specified measurement range is not included in the generation region R2 (step S7; No), the output control function 353 does not display a warning. Then, the output control function 353 displays the measurement result obtained by measuring the measurement range using the measurement function 354 (step S9).

[0050] The measurement function 354 determines whether or not it has received an operation to terminate the measurement (step S10). If it has not received an operation to terminate the measurement (step S10; No), the measurement function 354 accepts the specification of the measurement range in step S6.

[0051] On the other hand, if the operation to terminate the measurement is received (step S10; Yes), the measurement function 354 terminates the measurement.

[0052] Based on the above, the medical information processing device 30 terminates the measurement operation process.

[0053] As described above, the medical information processing device 30 according to the first embodiment inputs medical image data to the generation model 21 and acquires the generated image data generated by the generation model 21. The medical information processing device 30 uses the generated image data generated by the generation model 21 to measure the measurement range. The medical information processing device 30 then outputs predetermined information if the generated region R2 in the generated image data includes at least a part of the measurement range. Because predetermined information is output, medical professionals can understand that the image generated by the generation model 21 includes the measurement range. Therefore, when medical professionals use the image, the medical information processing device 30 makes it easy for them to understand that the image or a part of the image was generated by the generation model 21.

[0054] Furthermore, as shown in Figure 3, the medical information processing device 30 can also display the likelihood of each sub-region using the likelihood notification image R3. Therefore, the medical information processing device 30 can allow medical professionals to understand the likelihood of the image generated by the generation model 21.

[0055] (Variation 1) As explained earlier, the output control function 353 displays a likelihood notification image R3 that numerically shows the likelihood of each sub-region of the generated region R2, as shown in Figure 3. However, the output control function 353 may display the likelihood by means other than numerical values.

[0056] Figure 5 shows an example of a generated image G2 in which the generated region R2 can be identified. The output control function 353 may display a likelihood notification image R3a that shows the likelihood of each subregion of the generated region R2 in color. For example, if the likelihood is divided into multiple stages, the output control function 353 may display a likelihood notification image R3a that shows each subregion in a color corresponding to the stage to which the likelihood of the subregion belongs.

[0057] Furthermore, the output control function 353 may display a likelihood notification image R3a in which the likelihood is represented by the intensity of the color. In addition, the output control function 353 may display a likelihood notification image R3a in which the likelihood is represented not only by color, but also by characters, marks, or other methods.

[0058] (Second embodiment) In the medical information processing system 1b according to the second embodiment, the medical information processing device 30b extracts the generation region R2 based on the difference between the input image data input to the generation model 21 and the generated image data output from the generation model 21.

[0059] Figure 6 shows an example of a medical information processing system 1b according to the second embodiment. The processing circuit 35b includes a prompt acquisition function 351, a model control function 352b, a difference extraction function 355, an output control function 353b, and a measurement function 354.

[0060] The prompt acquisition function 351 and the measurement function 354 have the same functions as in the first embodiment.

[0061] The model control function 352b inputs the prompts obtained by the prompt acquisition function 351 to the generation model 21. The model control function 352b also acquires the generated image data generated by the generation model 21 by inputting the prompts.

[0062] The difference extraction function 355 extracts a generated region R2, which indicates the area where an image was generated by the generation model 21, based on the difference between the input image data input to the generation model 21 and the generated image data output from the generation model 21. For example, the difference extraction function 355 performs alignment between the input image data and the generated image data based on feature points such as anatomical landmarks. The difference extraction function 355 also calculates the difference between each pixel at the corresponding position in each aligned image. The difference extraction function 355 then extracts pixels with a difference greater than or equal to a threshold. Here, pixels with a difference greater than or equal to a threshold are considered to be regions generated by the generation model 21, and the difference extraction function 355 extracts the generated region R2. That is, the difference extraction function 355 extracts the shape of the generated region R2 and the position of the generated region R2.

[0063] The output control function 353b displays a generated image G2 on the display 34 that allows the generation region R2 to be identified, based on the difference extracted by the difference extraction function 355. In other words, the output control function 353b displays a generated image G2 that allows the generation region R2 to be identified without using image generation likelihood information.

[0064] Figure 7 shows an example of a generated image G2 that can identify the generated region R2. For example, the output control function 353b displays a generated notification image R4 with the same shape as the generated region R2 on the generated region R2 based on the difference extracted by the difference extraction function 355. The generated notification image R4 is an image that notifies the shape of the generated region R2 and the location of the generated region R2. The generated notification image R4 does not notify the likelihood of each subregion. However, healthcare professionals can understand from the generated notification image R4 that the image was generated by the generation model 21.

[0065] The output control function 353b displays the region in a manner that allows identification of the region where the image was generated by the generation model 21. Here, the generation model 21 generates an image of the image generation region R1. Then, the output control function 353b displays the generation region R2, which is the region of the image generated by the generation model 21, in a manner that allows identification.

[0066] As described above, the medical information processing device 30b according to the second embodiment issues a warning if the generated region R2 generated by the generation model 21 in the generated image data includes the measurement range. Medical professionals can recognize that the image was generated by the generation model 21 because they receive a warning. Therefore, the medical information processing device 30 makes it easy for medical professionals to recognize that the image or a part of the image was generated by the generation model 21 when they use the image.

[0067] Furthermore, as shown in Figure 7, the medical information processing device 30c displays a generation notification image R4 that notifies the shape of the generated region R2 and the location of the generated region R2. Therefore, the medical information processing device 30 can allow medical professionals to understand the region generated by the generation model 21.

[0068] (Third embodiment) In the third embodiment, the medical information processing device 30c of the medical information processing system 1c issues a warning if the generation area R2 includes the measurement range.

[0069] Figure 8 shows an example of a medical information processing system 1c according to the third embodiment. The processing circuit 35c includes a prompt acquisition function 351, a model control function 352b, a difference extraction function 355, an output control function 353c, and a measurement function 354c.

[0070] The prompt acquisition function 351, the model control function 352b, and the difference extraction function 355 have the same functions as in the second embodiment. Note that the processing circuit 35c shown in Figure 8 may also include a model control function 352 having the same functions as in the first embodiment. Furthermore, the processing circuit 35c does not necessarily have the difference extraction function 355.

[0071] The output control function 353c displays a generated notification image R4 with the same shape as the generated region R2 on the generated region R2, based on the difference between the input image data extracted by the difference extraction function 355 and the generated image data.

[0072] Figure 9 shows an example of measurement using the generated image G2. The measurement function 354c accepts an operation to specify the measurement range in the generated image G2. For example, the measurement function 354c accepts an operation to specify the range including the generated region R2 as the measurement range. For example, the measurement function 354c measures the measurement range specified by a measurement tool called caliper G5 shown in Figure 9. For example, the measurement function 354c measures the distance from one end of caliper G5 to the other end.

[0073] The output control function 353c displays a measurement result image G6 that shows the specified measurement range when the measurement range is specified by the measurement function 354c. For example, the measurement result image G6 is an image that shows the distance from one end to the other end of the caliper G5.

[0074] Here, the caliper G5 shown in Figure 9 measures the measurement range which includes the generation region R2 highlighted by the generation notification image R4. The output control function 353c displays a warning image G7 if the measurement range of the measurement function 354c is included in the generation region R2. The warning image G7 is an image that warns that the measurement range of the measurement function 354c is included in the generation region R2. Healthcare professionals can become aware that the generation region R2 is being measured when the warning image G7 is displayed.

[0075] As shown in Figure 9, the output control function 353c displays a generation notification image R4 that makes the generation region R2 identifiable. However, the output control function 353c may also display likelihood notification images R3 and R3a instead of the generation notification image R4. In this case as well, if the measurement range of the measurement function 354c is included in the generation region R2, the output control function 353c displays a warning image G7.

[0076] As described above, the medical information processing device 30c according to the third embodiment displays a warning image G7 when the generated region R2 generated by the generation model 21 in the generated image data includes the measurement range. Medical professionals can recognize that the image was generated by the generation model 21 in order to receive a warning. Therefore, the medical information processing device 30c makes it easy for medical professionals to recognize that the image was generated by the generation model 21 when they use the image.

[0077] (Variation 1) The output control functions 353, 353b, and 353c display the instruction information contained in the prompt on the display 34.

[0078] Figure 10 shows an example of measurement using the generated image G2. As shown in Figure 10, the output control functions 353, 353b, and 353c display a prompt image G8 that indicates a prompt instructing the generation model 21 to generate generated image data. For example, the output control functions 353, 353b, and 353c display the prompt image G8 in a position that does not overlap with the generation region R2. This ensures that the output control functions 353, 353b, and 353c do not prevent medical professionals from viewing the prompt image G8.

[0079] The prompt image G8 is an image that shows the instruction content information included in the prompt that instructs the generation of generated image data. In other words, the output control functions 353, 353b, and 353c display the instruction content for the generation model 21 indicated by the instruction content information acquired by the prompt acquisition function 351. For example, the prompt image G8 is an image that has text indicating the instruction content for the generation model 21, such as "Generate generated image data with artifacts present in the lung field region of the input image data reduced."

[0080] Here, prompt image G8 includes the phrase "artifact present in the lung field region." In other words, prompt image G8 contains artifact location information indicating the location where the artifact exists. Healthcare professionals can understand the location of the generated region R2 from prompt image G8. Note that prompt image G8 does not necessarily have to include artifact location information, for example, by saying "Generate generated image data with artifacts present in the input image data reduced." Furthermore, prompt image G8 may notify the location of the generated region R2 not only in relation to the subject's body part, but also by other representation methods such as coordinates.

[0081] Furthermore, the output control functions 353, 353b, and 353c may change the display format of the warning image G7 based on the instruction information contained in the prompt. For example, if the artifact location information contained in the instruction information differs from the location where the generation region R2 exists, the generation model 21 may have made a mistake in setting the image generation region R1. Therefore, the output control functions 353, 353b, and 353c highlight the warning image G7 when the artifact location information contained in the instruction information differs from the location where the generation region R2 exists. For example, the output control functions 353, 353b, and 353c display the warning image G7 by changing its color. Note that the output control functions 353, 353b, and 353c may not only change the color of the warning image G7 as a way to highlight it, but may also make it blink, add an icon, or display it in any other way.

[0082] As shown in Figure 10, the output control functions 353, 353b, and 353c display a generation notification image R4 that makes the generation region R2 identifiable. However, the output control functions 353, 353b, and 353c may display likelihood notification images R3 and R3a instead of the generation notification image R4. In this case as well, the output control functions 353, 353b, and 353c display a warning image G7 if at least a portion of the measurement range of the measurement functions 354 and 354c is included in the generation region R2.

[0083] (Modification 2) Output control functions 353, 353b, and 353c indicate that the measurement range is within the normal range when the measurement range is not included in the generation region R2.

[0084] Figure 11 shows an example of measurement using the generated image G2. As shown in Figure 11, the output control functions 353, 353b, and 353c display a normal image G9 indicating that the measurement range is normal when the generated region R2 does not include the measurement range of the measurement functions 354 and 354c. In other words, the normal image G9 is an image that notifies that the measurement range is not included in the image generated by the generation model 21. For example, the output control functions 353, 353b, and 353c display the normal image G9 when it is not within the measurement range using the caliper G5.

[0085] As shown in Figure 11, the output control functions 353, 353b, and 353c display a generation notification image R4 that makes the generation region R2 identifiable. However, the output control functions 353, 353b, and 353c may also display likelihood notification images R3 and R3a instead of the generation notification image R4. In this case as well, if the measurement range of the measurement functions 354 and 354c is not included in the generation region R2, the output control functions 353, 353b, and 353c display a normal image G9.

[0086] (Variation 3) Measurement functions 354 and 354c restrict the measurement of the measurement range if at least a portion of the measurement range is included in the generation region R2.

[0087] Figure 12 shows an example of measurement using the generated image G2. Measurement functions 354 and 354c restrict the measurement of the measurement range if at least a portion of the measurement range is included in the generated region R2. For example, as a measurement restriction, measurement functions 354 and 354c do not measure the measurement range.

[0088] Furthermore, measurement limitations are not limited to not measuring within the measurement range. For example, measurement functions 354 and 354c may limit measurements according to the required accuracy. For instance, measurement functions 354 and 354c may perform measurements in centimeter units but not in millimeter units. Also, measurement functions 354 and 354c may limit measurements according to the purpose and type of inspection.

[0089] The output control functions 353, 353b, and 353c display a measurement failure image G61 when the measurement functions 354 and 354c determine that the measurement range should not be measured. The measurement failure image G61 is an image that notifies that measurement is impossible because at least a part of the measurement range is included in the generation region R2.

[0090] (Modification 4) Measurement functions 354 and 354c restrict the measurement of the measurement range if the likelihood of a subregion of the generated region R2 that includes the measurement range is below a threshold.

[0091] Figure 13 shows an example of measurement using the generated image G2. The model control function 352 acquires image generation likelihood information, which has the likelihood of the image generated by the generation model 21, for each of the multiple subregions included in the generation region R2. The model control function 352 is an example of a likelihood acquisition unit. The measurement functions 354 and 354c restrict the measurement of the measurement range if the likelihood of the subregion of the generation region R2 that includes the measurement range is below a threshold. That is, the measurement functions 354 and 354c restrict the measurement of the measurement range if the likelihood of the subregion of the image generated by the generation model 21 is below a threshold. For example, as a restriction on measurement, the measurement functions 354 and 354c do not measure the measurement range.

[0092] Furthermore, measurement limitations are not limited to not measuring within the measurement range. For example, measurement functions 354 and 354c may limit measurements according to the required accuracy. For instance, measurement functions 354 and 354c may perform measurements in centimeter units but not in millimeter units. Also, measurement functions 354 and 354c may limit measurements according to the purpose and type of inspection.

[0093] On the other hand, measurement functions 354 and 354c measure the measurement range when the likelihood of the subregion containing the measurement range in the generation region R2 is above a threshold. In other words, measurement functions 354 and 354c measure the measurement range when the likelihood of the subregion of the image generated by the generation model 21 is above a threshold.

[0094] The output control functions 353, 353b, and 353c display a measurement impossible image G61 when the measurement functions 354 and 354c determine that the measurement range should not be measured. The measurement impossible image G61 is an image that notifies that measurement will not be performed because the measurement range includes an area with a low likelihood of measurement.

[0095] On the other hand, the output control functions 353, 353b, and 353c display the normal image G9 when the measurement functions 354 and 354c determine that the measurement range has been measured.

[0096] Figure 14 shows an example of measurement using the generated image G2. As shown in Figure 14, the output control functions 353, 353b, and 353c display a normal image G9 to indicate that the measurement range is normal. Alternatively, the output control functions 353, 353b, and 353c may also display a normal image G9 to indicate that the measurement range is normal because the likelihood of a subregion of the generated region R2 that includes the measurement range is above a threshold.

[0097] (Variation 5) Output control functions 353, 353b, and 353c display warnings not only for 2D medical image data but also for 3D medical image data.

[0098] Figure 15 shows an example of measurement using three-dimensional medical image data. The model control function 352 acquires three-dimensional medical image data in which at least one slice image is the generated image data. That is, the model control function 352 acquires three-dimensional medical image data including the generated image data generated by the generation model 21. In addition, if image generation likelihood information is output from the generation model 21, the model control function 352 may acquire the image generation likelihood information.

[0099] The measurement function 354 accepts an operation to specify a measurement range. In this case, the measurement function 354 may accept a specification of a measurement range that includes multiple slice images. That is, the measurement functions 354 and 354c measure the three-dimensional measurement range included in the three-dimensional generated region notification image data.

[0100] Output control functions 353, 353b, and 353c add identifier G31 to the 2D generated image data containing the subregion with a likelihood lower than the threshold when the 3D measurement range from the measurement functions 354 and 354c is included in the subregion with a likelihood lower than the threshold, and display it. This allows medical professionals to understand whether the measurement range is included in the image generated by the generation model 21, even in the 3D generated region notification image data.

[0101] (Experimental variation 6) It was explained that the output control functions 353, 353b, and 353c output predetermined information when the generation region R2 of the generated image data generated by the generation model 21 includes the measurement range measured by the measurement functions 354 and 354c. However, the output control functions 353, 353b, and 353c may output predetermined information not only when the generation region R2 includes the measurement range, but also when the generation region R2 is used in other ways. For example, the output control functions 353, 353b, and 353c may output predetermined information when an arrow pointing to the generation region R2 is added, when the region including the generation region R2 is highlighted with color, or when it is captured in an image interpretation report.

[0102] (Example 7) The model control functions 352 and 352b are described as inputting input image data into the generation model 21 and acquiring the generated image data generated by the generation model 21. The data input to the generation model 21 is not limited to CT image data generated by the reconstruction process, but may also be data before the reconstruction process. The data before the reconstruction process may be raw data detected by the X-ray detector, or projection data that has undergone preprocessing such as logarithmic transformation, offset correction, inter-channel sensitivity correction, and beam hardening correction. In this case as well, the output control functions 353, 353b, and 353c input the data before the reconstruction process into the generation model 21, and output predetermined information if the generation region R2 in the generated image data generated by the generation model 21 includes at least a part of the measurement range.

[0103] According to at least one embodiment described above, when a medical professional performs measurements using an image, it is possible to easily make it clear to the medical professional that the image or a part of the image was generated by the generation model 21.

[0104] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0105] 1, 1b, 1c Medical Information Processing System 10. Medical Information Management System 20 Generative Model Server 21 Generative Models 30, 30b, 30c Medical Information Processing Devices 35, 35b, 35c Processing circuits 351 Prompt acquisition function 352, 352b Model control function 353, 353b, 353c Output control function 354, 354c measurement function A1 Artifact Region G1 Input Image G2 generated image G5 Caliper G6 Measurement Results Image G61 Image not measurable G7 warning image G8 prompt image G9 Normal Image G31 identifier R1 Image generation region R2 generation area R3, R3a Likelihood Notification Image R4 Generation Notification Image

Claims

1. An image acquisition unit that inputs medical image data into a generation model and acquires the generated image data generated by the generation model, A measurement unit that measures the measurement range using the generated image data, A control unit that outputs predetermined information when the generation region in the generated image data includes at least a part of the measurement range, A medical information processing device equipped with [a specific feature].

2. The generation region of the generated image data includes the image data generated by the generation model. The medical information processing device according to claim 1.

3. The control unit, if the generated region in the generated image data includes at least a portion of the measurement range, notifies the measurement unit to perform a remeasurement. The medical information processing device according to claim 1.

4. The generated region of the generated image data is at least one of the following: a region in the medical image data where artifacts have been reduced; a region in the medical image data where information outside the imaging region has been generated; and a region in the medical image data where devices related to treatment have been removed. The medical information processing device according to claim 1.

5. The control unit displays the likelihood of the image generated by the generation model for each of the multiple sub-regions included in the generation region. The medical information processing device according to claim 1.

6. The control unit displays a prompt instructing the generation model to generate the generated image data. The medical information processing device according to claim 5.

7. The control unit displays an image indicating that the measurement range is normal if the generation area does not include the measurement range. The medical information processing device according to claim 1.

8. The measurement unit shall not measure the measurement range if the generation region includes the measurement range. The medical information processing device according to claim 1.

9. The system further includes a likelihood acquisition unit that acquires the likelihood of the image generated by the generation model for each of the multiple sub-regions included in the generation region, The measurement unit shall not measure the measurement range if the measurement range is included in the sub-region of the likelihood below the threshold. The medical information processing device according to claim 1.

10. The measurement unit measures the measurement range when the likelihood of the sub-region in the generation region that includes the measurement range is greater than or equal to a threshold. The medical information processing device according to claim 9.

11. An image acquisition unit that inputs medical image data into a generation model and acquires the generated image data output from the generation model, A control unit that displays the generated image data in a manner that allows identification of the generated region in the generated image data as a region where an image was generated by the generation model, A medical information processing device equipped with [a specific feature].

12. A medical information processing system comprising a generation model that generates at least a portion of medical image data, and a medical information processing device that displays the generated image data output from the generation model, The aforementioned medical information processing device is An image acquisition unit that inputs medical image data into a generation model and acquires the generated image data output from the generation model, A measurement unit that measures the measurement range using the generated image data, The system includes a control unit that outputs predetermined information when the generation region in the generated image data includes at least a portion of the measurement range. Medical information processing system.

13. Medical image data is input into a generation model, and the generated image data output from the generation model is obtained. Using the generated image data, the measurement range is measured. If the generation region in the generated image data includes at least a portion of the measurement range, predetermined information is output. A medical information processing program that causes a computer to perform certain tasks.

14. Medical image data is input into a generation model, and the generated image data output from the generation model is obtained. Using the generated image data, the measurement range is measured. If the generation region in the generated image data includes at least a portion of the measurement range, predetermined information is output. A medical information processing method including [the specified term].

Images