Medical information processing apparatus, x-ray diagnostic apparatus, and storage medium

Abstract

Embodiments relate to a medical information processing apparatus, an X-ray diagnostic apparatus, and a medical information processing program. In order to more effectively utilize a medical image, a medical information processing apparatus of an embodiment is provided with an image acquisition unit, an event acquisition unit, a management unit, and an output unit. The image acquisition unit acquires medical images in order in a treatment procedure on a subject. The event acquisition unit acquires events in the treatment procedure in order based on the medical images in the treatment procedure. The management unit manages the medical images and the events in association with time information in the treatment procedure. The output unit outputs the medical images in a manner in which a relationship with the events can be recognized.

Description

[0001] Reference to relevant applications

[0002] This application claims priority to Japanese Patent Application No. 2020-128499, filed on July 29, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The implementation methods involve medical information processing devices, X-ray diagnostic devices, and medical information processing programs. Background Technology

[0004] During surgical procedures on a patient, medical images collected during the procedure are displayed to assist the physician or other user performing the procedure. These medical images are searched by the user or other staff from a large number of medical images collected during the procedure. Summary of the Invention

[0005] The technical problem solved by this invention is to utilize medical images more effectively.

[0006] The medical information processing apparatus of this embodiment includes an image acquisition unit, an event acquisition unit, a management unit, and an output unit. The image acquisition unit sequentially acquires medical images during a treatment procedure on a patient. The event acquisition unit, based on the medical images, sequentially acquires events occurring during the treatment procedure. The management unit manages the medical images and events by establishing associations with time information during the treatment procedure. The output unit outputs the medical images in a manner that allows identification of their relationship to the events.

[0007] Effect

[0008] The ultrasound diagnostic device according to the embodiment can utilize medical images more effectively. Attached Figure Description

[0009] Figure 1 This is a block diagram illustrating an example of the structure of the medical information processing system according to the first embodiment.

[0010] Figure 2 This is a block diagram illustrating an example of the structure of the X-ray diagnostic apparatus according to the first embodiment.

[0011] Figure 3A This diagram illustrates an example of the event acquisition and processing in the first embodiment.

[0012] Figure 3B This diagram illustrates an example of the event acquisition and processing in the first embodiment.

[0013] Figure 3C This diagram illustrates an example of the event acquisition and processing in the first embodiment.

[0014] Figure 4 This diagram illustrates an example of the event acquisition and processing in the first embodiment.

[0015] Figure 5 This is a diagram showing an example of the first embodiment.

[0016] Figure 6 This diagram illustrates an example of the associated image acquisition process in the first embodiment.

[0017] Figure 7 This is a diagram showing an example of the first embodiment.

[0018] Figure 8 This is a diagram illustrating the hierarchical display of events in the first embodiment.

[0019] Figure 9A This is a diagram showing an example of the first embodiment.

[0020] Figure 9B This is a diagram showing an example of the first embodiment.

[0021] Figure 10A This is a diagram showing an example of the first embodiment.

[0022] Figure 10B This is a diagram showing an example of the first embodiment.

[0023] Figure 11 This is a diagram illustrating an example of the second embodiment.

[0024] Figure 12 This is a diagram illustrating a display example of the third embodiment.

[0025] Figure 13 This is a diagram showing an example of the fourth embodiment. Detailed Implementation

[0026] Hereinafter, with reference to the accompanying drawings, the implementation methods of the medical information processing device, the X-ray diagnostic device, and the medical information processing program will be described in detail.

[0027] (First Implementation)

[0028] In the first embodiment, a medical information processing system 1 including a medical information processing device 30 will be described as an example. Furthermore, in the first embodiment, an X-ray image collected by an X-ray diagnostic device 10 will be described as an example of a medical image. For example, the medical information processing system 1... Figure 1 As shown, it includes an X-ray diagnostic device 10, an image storage device 20, and a medical information processing device 30. Additionally, Figure 1This is a block diagram illustrating an example of the structure of the medical information processing system 1 according to the first embodiment.

[0029] like Figure 1 As shown, the X-ray diagnostic apparatus 10, image storage device 20, and medical information processing device 30 are connected via a network NW. Here, the network NW can be either a local area network (LAN) closed within the hospital or a network connected via the Internet. That is, the image storage device 20, the X-ray diagnostic apparatus 10, and the medical information processing device 30 can be located in the same facility or in different facilities. However, the medical information processing device 30 is typically located in the examination room where the X-ray diagnostic apparatus 10 is installed, or in the operating room for operating the X-ray diagnostic apparatus 10.

[0030] The X-ray diagnostic apparatus 10, for example, sequentially collects X-ray images during a treatment procedure on a subject P, and transmits the collected X-ray images to an image storage device 20 or a medical information processing device 30. Further details regarding the X-ray diagnostic apparatus 10 will be provided later.

[0031] Image storage device 20 stores various medical images. For example, image storage device 20 receives and stores X-ray images collected by X-ray diagnostic device 10. For example, image storage device 20 is a server for PACS (Picture Archiving and Communication System).

[0032] The medical information processing device 30 performs various processes based on X-ray images acquired from the X-ray diagnostic device 10 or the image storage device 20. For example, the medical information processing device 30... Figure 1 As shown, it has an input interface 31, a display 32, a memory 33 and a processing circuit 34.

[0033] Input interface 31 accepts various input operations from the user, converts the accepted input operations into electrical signals, and outputs them to processing circuit 34. For example, input interface 31 can be implemented using a mouse, keyboard, trackball, switch, button, joystick, touchpad for input operations via a touch operating surface, touchscreen integrating the display screen and touchpad, contactless input circuit using an optical sensor, or voice input circuit. Alternatively, input interface 31 can also be a tablet terminal capable of wireless communication with the main body of medical information processing device 30. Furthermore, input interface 31 can also be a circuit that accepts user input operations through motion capture. For example, input interface 31 can process signals obtained via a tracker and images collected from the user, thereby accepting the user's body movements, gaze, etc., as input operations. Furthermore, input interface 31 is not limited to interfaces with physical operating components such as mice and keyboards. For example, processing circuits that receive electrical signals corresponding to input operations from external input devices separately from medical information processing device 30 and output these electrical signals to processing circuit 34 are also included in the example of input interface 31.

[0034] Display 32 displays various information. For example, under the control of processing circuit 34, display 32 displays X-ray images collected during the treatment procedure on subject P. The display of medical images on display 32 will be described later. Additionally, display 32 displays a GUI (Graphical User Interface) for receiving various instructions and settings from the user via input interface 31. For example, display 32 may be a liquid crystal display (LCD) or a CRT (Cathode Ray Tube) display. Display 32 can be desktop or a tablet terminal capable of wireless communication with the main body of medical information processing device 30.

[0035] In addition, Figure 1 In this description, the medical information processing device 30 is assumed to have a display 32, but the medical information processing device 30 may also replace the display 32 or include a projector. The projector can project onto a screen, wall, floor, or the body surface of the subject P under the control of the processing circuit 34. For example, the projector can also project onto any plane, object, or space via projection mapping. Furthermore, the medical information processing device 30 may replace the display 32 or, based on the display 32, display the image as the object on the display 108 of the X-ray diagnostic device 10.

[0036] The memory 33 may be implemented using semiconductor memory elements such as RAM (Random Access Memory), flash memory, hard disks, or optical disks. For example, the memory 33 stores programs used by the circuitry included in the medical information processing device 30 to perform its functions. Additionally, the memory 33 stores X-ray images acquired from the X-ray diagnostic device 10 or the image storage device 20. Alternatively, the memory 33 may be implemented using a server cluster (cloud) connected to the medical information processing device 30 via a network NW.

[0037] The processing circuit 34 controls the overall operation of the medical information processing device 30 by executing control function 34a, image acquisition function 34b, event acquisition function 34c, management function 34d, and output function 34e. Here, image acquisition function 34b is an example of an image acquisition unit. Event acquisition function 34c is an example of an event acquisition unit. Management function 34d is an example of a management unit. Output function 34e is an example of an output unit.

[0038] For example, the processing circuit 34 reads from the memory 33 and executes the program corresponding to the control function 34a, thereby controlling various functions such as the image acquisition function 34b, the event acquisition function 34c, the management function 34d, and the output function 34e based on various input operations received from the user via the input interface 31.

[0039] Additionally, for example, processing circuit 34 reads from memory 33 and executes a program corresponding to image acquisition function 34b, thereby sequentially acquiring X-ray images during the treatment procedure on subject P. Additionally, for example, processing circuit 34 reads from memory 33 and executes a program corresponding to event acquisition function 34c, thereby sequentially acquiring events during the treatment procedure based on the X-ray images. Additionally, for example, processing circuit 34 reads from memory 33 and executes a program corresponding to management function 34d, thereby managing the X-ray images and events in association with time information during the treatment procedure. Additionally, for example, processing circuit 34 reads from memory 33 and executes a program corresponding to output function 34e, thereby outputting the X-ray images in a manner that allows for identification of their relationship with events. Furthermore, the processing based on image acquisition function 34b, event acquisition function 34c, management function 34d, and output function 34e will be described later.

[0040] exist Figure 1 In the medical information processing device 30 shown, each processing function is stored in the memory 33 as a program that can be executed by a computer. The processing circuit 34 is a processor that implements the function corresponding to each program by reading it from the memory 33 and executing the program. In other words, the processing circuit 34, having read the state of the program, has the function corresponding to the read program.

[0041] In addition, Figure 1 In this description, the control function 34a, image acquisition function 34b, event acquisition function 34c, management function 34d, and output function 34e are implemented by a single processing circuit 34. However, it is also possible to describe the process circuit 34 as a combination of multiple independent processors, with each processor executing a program to implement the functions. In addition, the processing functions of the processing circuit 34 can also be appropriately distributed or combined into one or more processing circuits.

[0042] Alternatively, the processing circuit 34 can also utilize the processor of an external device connected via the network NW to implement its functions. For example, the processing circuit 34 reads and executes programs corresponding to each function from the memory 33, and uses a server cluster (cloud) connected to the medical information processing device 30 via the network NW as a computing resource, thereby achieving... Figure 1 The functions shown.

[0043] Next, use Figure 2 The X-ray diagnostic device 10 will be described. Figure 2 This is a block diagram illustrating an example of the structure of the X-ray diagnostic apparatus 10 according to the first embodiment. For example... Figure 2 As shown, the X-ray diagnostic device 10 includes an X-ray high-voltage device 101, an X-ray tube 102, an X-ray aperture 103, a top plate 104, a C-arm 105, an X-ray detector 106, a memory 107, a display 108, an input interface 109, and a processing circuit 110.

[0044] Under the control of the processing circuit 110, the X-ray high-voltage device 101 supplies high voltage to the X-ray tube 102. For example, the X-ray high-voltage device 101 includes: a high-voltage generating device with circuitry such as a transformer and rectifier to generate the high voltage applied to the X-ray tube 102; and an X-ray control device to control the output voltage corresponding to the X-rays irradiated by the X-ray tube 102. Furthermore, the high-voltage generating device can be a transformer or an inverter.

[0045] X-ray tube 102 is a vacuum tube having a cathode (filament) that generates thermionic electrons and an anode (target) that generates X-rays by collisions with thermionic electrons. X-ray tube 102 generates X-rays by irradiating thermionic electrons from the cathode toward the anode using a high voltage supplied from X-ray high-voltage device 101.

[0046] The X-ray aperture 103 has a collimator that narrows the irradiation range of the X-rays generated by the X-ray tube 102 and a filter that adjusts the X-rays irradiated from the X-ray tube 102.

[0047] The collimator in the X-ray aperture 103 has, for example, four sliding aperture blades. By sliding the aperture blades, the collimator concentrates the X-rays generated by the X-ray tube 102 and irradiates the subject P. Here, the aperture blades are plate-shaped components made of lead or the like, and are positioned near the X-ray irradiation port of the X-ray tube 102 to adjust the irradiation range of the X-rays.

[0048] The filter in the X-ray aperture 103 aims to reduce the radiation dose to the subject P and improve the image quality of the X-ray image data. Depending on its material and thickness, the filter alters the linear quality of the transmitted X-rays, reducing soft-ray components easily absorbed by the subject P, or reducing high-energy components that reduce the contrast of the X-ray image data. Furthermore, the filter, based on its material, thickness, and position, alters the X-ray dose and irradiation range, attenuating the X-rays to ensure a predetermined distribution of X-rays irradiating the subject P from the X-ray tube 102.

[0049] For example, the X-ray aperture 103 has a drive mechanism such as a motor and actuator. Under the control of the processing circuit 110 (described later), the drive mechanism is activated, thereby controlling the X-ray irradiation. For example, the X-ray aperture 103 applies a drive voltage to the drive mechanism according to a control signal received from the processing circuit 110, thereby adjusting the opening of the collimator's aperture blades and controlling the irradiation range of the X-rays irradiating the subject P. Furthermore, for example, the X-ray aperture 103 applies a drive voltage to the drive mechanism according to a control signal received from the processing circuit 110, thereby adjusting the position of the filter and controlling the dose distribution of the X-rays irradiating the subject P.

[0050] The top plate 104 is a bed on which the subject P is placed, positioned above an examination bed (not shown). The subject P is not included in the X-ray diagnostic apparatus 10. For example, the examination bed has a drive mechanism such as a motor and actuator, which, under the control of the processing circuit 110 (described later), actuates the drive mechanism, thereby controlling the movement and tilting of the top plate 104. For example, the examination bed applies a drive voltage to the drive mechanism based on a control signal received from the processing circuit 110, thereby causing the top plate 104 to move or tilt.

[0051] The C-arm 105 holds the X-ray tube 102, X-ray aperture 103, and X-ray detector 106 facing each other across the subject P. For example, the C-arm 105 has a drive mechanism such as a motor and actuator, which, under the control of the processing circuit 110 (described later), rotates or moves. For example, the C-arm 105 applies a drive voltage to the drive mechanism according to a control signal received from the processing circuit 110, thereby causing the X-ray tube 102, X-ray aperture 103, and X-ray detector 106 to rotate and move relative to the subject P, controlling the X-ray irradiation position and angle. Furthermore, in Figure 2The example given is a single-plane X-ray diagnostic device 10, but the implementation is not limited to this and can also be a dual-plane device.

[0052] The X-ray detector 106 is, for example, an X-ray flat panel detector (FPD) having detection elements arranged in a matrix. The X-ray detector 106 detects X-rays that have irradiated and transmitted through the X-ray tube 102 onto the subject P, and outputs a detection signal corresponding to the detected X-ray amount to the processing circuit 110. Alternatively, the X-ray detector 106 can be an indirect conversion type detector having a grid, a scintillator array, and a photosensitive sensor array, or a direct conversion type detector having semiconductor elements that convert the incident X-rays into electrical signals.

[0053] The memory 107 may be implemented using semiconductor memory elements such as RAM or flash memory, hard disks, or optical disks. For example, the memory 107 stores programs corresponding to various functions that are read and executed by the processing circuit 110. Furthermore, the memory 107 may also be implemented via the cloud.

[0054] The display 108 displays various information. For example, under the control of the processing circuit 110, the display 108 displays a GUI for receiving user instructions and various X-ray images. For example, the display 108 is a liquid crystal display or a CRT display. In addition, the display 108 can be a desktop computer or a tablet terminal or the like capable of wireless communication with the processing circuit 110.

[0055] In addition, Figure 2 In this description, the X-ray diagnostic apparatus 10 is shown to have a display 108, but the X-ray diagnostic apparatus 10 may also replace the display 108 or include a projector. The projector can project onto a screen, wall, floor, or the body surface of the subject P under the control of the processing circuit 110. For example, the projector can also project onto any plane, object, or space through projection mapping.

[0056] Input interface 109 accepts various input operations from the user, converts the accepted input operations into electrical signals, and outputs them to processing circuit 110. For example, input interface 109 can be implemented using a mouse, keyboard, trackball, switch, button, joystick, touchpad for input via a touch operating surface, touchscreen integrating the display screen and touchpad, contactless input circuit using an optical sensor, or voice input circuit. Furthermore, input interface 109 can also be configured as a tablet terminal capable of wireless communication with processing circuit 110. Additionally, input interface 109 can also be a circuit that accepts user input operations through motion capture. For example, input interface 109 processes signals obtained via a tracker and images collected by the user, thereby accepting user movements, gaze, etc., as input operations. Furthermore, input interface 109 is not limited to interfaces with physical operating components such as mice and keyboards. For example, a processing circuit that receives an electrical signal corresponding to an input operation from an external input device that is separately installed from the X-ray diagnostic apparatus 10, and outputs the electrical signal to the processing circuit 110, is also included in the example of the input interface 109.

[0057] The processing circuit 110 controls the overall operation of the X-ray diagnostic device 10 by executing control function 110a, collection function 110b and output function 110c.

[0058] For example, the processing circuit 110 reads from the memory 107 and executes the program corresponding to the control function 110a, thereby controlling various functions such as the collection function 110b and the output function 110c from various input operations received by the user via the input interface 109.

[0059] For example, the processing circuit 110 reads from the memory 107 and executes a program equivalent to the collection function 110b, thereby collecting X-ray images from the subject P. For example, during a treatment procedure on the subject P, the collection function 110b sequentially collects X-ray images based on the X-rays transmitted through the subject P.

[0060] For example, the collection function 110b controls the X-ray high-voltage device 101, adjusting the voltage supplied to the X-ray tube 102, thereby controlling the amount of X-rays irradiated onto the subject P and its on / off state. Additionally, the collection function 110b controls the operation of the X-ray aperture 103, adjusting the opening of the aperture blades in the collimator, thereby controlling the irradiation range of the X-rays irradiating the subject P. Furthermore, the collection function 110b controls the operation of the X-ray aperture 103, adjusting the position of the filter, thereby controlling the dose distribution of the X-rays. Additionally, the collection function 110b controls the operation of the C-arm 105, changing the position and angle of the C-arm 105 relative to the subject P, thereby controlling the imaging position and angle. Furthermore, the position and angle of the C-arm 105 are also recorded as arm position information. Finally, the collection function 110b generates an X-ray image based on the detection signal received from the X-ray detector 106 and stores the generated X-ray image in the memory 107.

[0061] Additionally, the processing circuit 110 reads from the memory 107 and executes a program equivalent to the output function 110c, thereby controlling the transmission and reception of data via the network NW and the display on the display 32. For example, the output function 110c transmits the X-ray images collected by the collection function 110b to the medical information processing device 30 via the network NW, or displays them on the display 32. Furthermore, for example, the output function 110c causes the display 32 to display a GUI for accepting input operations from the user.

[0062] exist Figure 2 In the X-ray diagnostic apparatus 10 shown, each processing function is stored in the memory 107 as a program executable by a computer. The processing circuit 110 is a processor that reads out and executes programs from the memory 107 to implement the functions corresponding to each program. In other words, the processing circuit 110, having read out the state of the program, has the functions corresponding to the read-out program.

[0063] In addition, Figure 2 The control function 110a, collection function 110b, and output function 110c are described using a single processing circuit 110. However, multiple independent processors can also be combined to form the processing circuit 110, with each processor executing a program to implement the functions. In addition, the processing functions of the processing circuit 110 can be appropriately distributed or combined into one or more processing circuits.

[0064] Alternatively, the processing circuit 110 can also utilize the processor of an external device connected via the network NW to implement its functions. For example, the processing circuit 110 reads and executes programs corresponding to each function from the memory 107, and uses a server cluster (cloud) connected to the X-ray diagnostic apparatus 10 via the network NW as a computing resource, thereby achieving... Figure 2 The functions shown.

[0065] The medical information processing system 1, which includes an X-ray diagnostic device 10, an image storage device 20, and a medical information processing device 30, has been described above. The processing performed in the medical information processing system 1 will now be described.

[0066] For example, during a surgical procedure on a patient P, the X-ray diagnostic device 10 sequentially collects X-ray images from the patient P. Additionally, the medical information processing device 30 sequentially acquires X-ray images from the X-ray diagnostic device 10 and displays these images sequentially and in real-time on the display 32. Thus, the user can smoothly perform the surgical procedure while simultaneously confirming the internal structure of the patient P and the medical devices inserted into the patient P. In other words, the medical information processing system 1 assists the user (attending physician) performing the surgical procedure by collecting and displaying X-ray images.

[0067] Here, most surgical procedures are performed by a team of multiple people, including the attending physician. Sometimes, members other than the attending physician may want to refer to real-time X-ray images to understand the current situation during the procedure. Additionally, during the execution of a surgical procedure, the attending physician or other members may refer to the collected X-ray images.

[0068] However, for example, it is not limited to depicting the unique features of the current situation during a surgical procedure in real-time X-ray images; there are also situations where it is difficult to determine the process leading to the current situation based solely on real-time X-ray images. Therefore, even when referring to real-time X-ray images during a surgical procedure, team members may not be able to grasp the current situation of the procedure, leading to difficulties in communication within the team. Furthermore, for example, during the execution of a surgical procedure, when team members want to display the collected X-ray images as reference images on the monitor 108, they may sometimes spend considerable time searching for the necessary images.

[0069] Therefore, the medical information processing device 30, through the processing circuit 34, can more effectively utilize medical images such as X-ray images. The processing performed by the processing circuit 34 will be described in detail below.

[0070] First, during the treatment procedure on the subject P, the image acquisition function 34b sequentially acquires X-ray images. Furthermore, the image acquisition function 34b can acquire X-ray images either via the image storage device 20 or directly from the X-ray diagnostic device 10. Additionally, the event acquisition function 34c sequentially acquires events occurring during the treatment procedure.

[0071] Here, use Figure 3A , Figure 3B and Figure 3C An example of event retrieval processing performed by event retrieval function 34c is illustrated. Figure 3A , Figure 3B and Figure 3C This diagram illustrates an example of the event acquisition and processing in the first embodiment.

[0072] First of all, Figure 3A Explanation will be provided. In Figure 3A In the illustrated case, the event acquisition function 34c accepts patient information input (step S101) before the start of the treatment procedure, and also accepts the selection of the procedure (step S102). For example, the event acquisition function 34c can execute steps S101 and S102 by accepting input operations from the user via the input interface 31. Alternatively, steps S101 and S102 can be omitted as appropriate.

[0073] Additionally, the event acquisition function 34c accepts the selection of the event stream (step S103). Figure 3A As an example, the case where an event stream containing events E1, E2, and E3 is selected will be explained. For example, the event retrieval function 34c can execute step S103 by accepting input from the user via input interface 31. For instance, the memory 33 may pre-store a database containing multiple types of event streams, and the event retrieval function 34c can select any event stream from this database upon receiving an input from the user. Alternatively, the event retrieval function 34c may automatically select an event stream based on the patient information received in step S101 and the surgery selected in step S102.

[0074] After step S103, the event acquisition function 34c determines that the treatment surgery is in the initial event E1 state (step S104). Additionally, after the treatment surgery begins, the image acquisition function 34b acquires the X-ray image collected in the X-ray diagnostic device 10 (step S105). Here, the event acquisition function 34c can acquire event E1 based on the X-ray image acquired in step S105.

[0075] Furthermore, the management function 34d manages the X-ray image and event E1 obtained in step S105 in association with the time information during the treatment procedure. For example, the management function 34d adds event E1 and time information to the X-ray image obtained in step S105 as supplementary information and stores it in the memory 33. For instance, the management function 34d stores event E1 and time information in the DICOM (Digital Imaging and Communications in Medicine) tag of the X-ray image. The time information can be either the date and time information of the X-ray image collection performed in the X-ray diagnostic device 10, or the date and time information of the medical information processing device 30 obtaining the X-ray image from the X-ray diagnostic device 10. Additionally, the management function 34d can further store patient information received in step S101, the procedure selected in step S102, etc., as supplementary information in the X-ray image.

[0076] Next, the event acquisition function 34c determines whether event E1 has ended (step S106). If it has not ended (step S106 negation), it proceeds to step S105 again. For example, the event acquisition function 34c automatically determines whether event E1 has ended based on the collected X-ray image. For example, if event E1 is "the medical device is inserted from the thigh of the subject P and reaches the heart of the treatment target site," the event acquisition function 34c can automatically determine whether event E1 has ended based on the positional relationship between the medical device and the heart of the subject P in the X-ray image. Then, if event E1 has not ended, the event acquisition function 34c proceeds to step S105 again to acquire event E1 for a newly acquired X-ray image. That is, the event acquisition function 34c can acquire the event based on the X-ray image. Therefore, even if the X-ray image itself does not depict the characteristics unique to event E1, the information of event E1 can be assigned to the X-ray image.

[0077] Alternatively, the event acquisition function 34c can automatically determine whether event E1 has ended by comparing the location information of the surrounding device with the database. For example, when the C-arm 105 moves in a manner that follows the position of the front end of the medical device, the event acquisition function 34c can automatically determine whether event E1 has ended based on the location information of the C-arm 105. Alternatively, the event acquisition function 34c can also determine whether event E1 has ended by accepting input from the user via the input interface 31.

[0078] On the other hand, if event E1 ends (affirmed in step S106), the event acquisition function 34c determines that the treatment procedure has moved to event E2 (step S107). Additionally, the image acquisition function 34b acquires the X-ray image collected in the X-ray diagnostic device 10 (step S108). Here, the event acquisition function 34c can acquire event E2 based on the X-ray image acquired in step S108. For example, the event acquisition function 34c determines whether event E1 has ended based on the X-ray image acquired in step S105; if it has ended, it determines that the treatment procedure has moved to event E2, and acquires event E2 based on the newly acquired X-ray image in step S108. Furthermore, the management function 34d manages the X-ray image acquired in step S108 and event E2 in association with the time information during the treatment procedure. Moreover, the determination of the end of event E1 in step S106 can be performed either by detecting features specific to the end of event E1 or by detecting features specific to event E2 to determine the start of event E2. The same method for determination can be applied in step S109, etc.

[0079] Next, the event acquisition function 34c determines whether event E2 has ended (step S109). If it has not ended (step S109 negation), it proceeds to step S108 again. For example, the event acquisition function 34c determines whether event E2 has ended based on the X-ray image. If it has not ended, it proceeds to step S108 again and acquires event E2 for the newly acquired X-ray image. On the other hand, if event E2 has ended (step S109 affirmation), the event acquisition function 34c determines that the treatment surgery has moved to event E3 (step S110). In addition, the image acquisition function 34b acquires the X-ray image collected in the X-ray diagnostic device 10 (step S111). Here, the event acquisition function 34c can acquire event E3 for the X-ray image acquired in step S111. For example, the event acquisition function 34c determines whether event E2 has ended based on the X-ray image acquired in step S108. If it has ended, it determines that the treatment surgery has moved to event E3, and acquires event E3 based on the newly acquired X-ray image in step S111. The management function 34d manages the X-ray image acquired in step S111 and event E3 in association with the time information in the treatment surgery.

[0080] Next, the event acquisition function 34c determines whether event E3 has ended (step S112). If it has not ended (step S112 negates), it proceeds to step S111 again. For example, the event acquisition function 34c determines whether event E3 has ended based on the X-ray image. If it has not ended, it proceeds to step S111 again and acquires event E3 for the newly acquired X-ray image. On the other hand, if event E3 has ended (step S112 affirms), the event acquisition function 34c ends the process.

[0081] That is, according to Figure 3A The processing described involves image acquisition function 34b sequentially acquiring X-ray images during the treatment procedure on the subject P, and event acquisition function 34c sequentially acquiring events during the treatment procedure. Specifically, event acquisition function 34c determines the images in the sequentially acquired X-ray images where events have occurred, based on the sequentially acquired X-ray images. For example, event acquisition function 34c can determine whether each event in the event stream has ended based on the X-ray images, thereby enabling it to sequentially acquire events for newly acquired X-ray images. Furthermore, management function 34d can manage X-ray images and events in association with time information during the treatment procedure. That is, management function 34d stores the acquired medical images and stores corresponding information about the events that have occurred in the stored medical images.

[0082] Next, regarding Figure 3B Explanation will be provided. In Figure 3B In the illustrated case, the event acquisition function 34c accepts patient information input (step S201) before the start of the treatment procedure, and also accepts the selection of the procedure (step S202). For example, the event acquisition function 34c can execute steps S201 and S202 by accepting input operations from the user via the input interface 31. Alternatively, steps S201 and S202 can be omitted as appropriate.

[0083] Additionally, the event acquisition function 34c sets the event name. For example, after step S202, the event acquisition function 34c sets the initial event name during the treatment procedure (step S203). For example, the event acquisition function 34c can set the event name by accepting input from the user via the input interface 31. As an example, the event acquisition function 34c can set the event name via voice input. Alternatively, the event acquisition function 34c can automatically set the event name based on the patient information received in step S201 and the procedure selected in step S202.

[0084] Next, the event acquisition function 34c determines whether to correct the event name (step S204). For example, in the event of a plan change or inconsistency between automatic identification and user identification, the event acquisition function 34c determines that the event name should be corrected (step S204 affirmative), and the event name can be corrected (step S205). Here, the event acquisition function 34c can either correct the event name by accepting input from the user or automatically reset the event name.

[0085] On the other hand, without modifying the event name (step S204 negation), the image acquisition function 34b acquires the X-ray image collected in the X-ray diagnostic apparatus 10 (step S206). Here, the event acquisition function 34c can acquire the event name set in step S203 or modified in step S205 for the X-ray image acquired in step S206. The management function 34d manages the X-ray images and events in association with the time information during the treatment procedure.

[0086] Next, the event acquisition function 34c determines whether the event with the event name set in step S203 or modified in step S205 has ended (step S207). If it has not ended (step S207 negation), it proceeds to step S206 again. For example, the event acquisition function 34c automatically determines whether the event has ended based on the collected X-ray image. Then, if the event has not ended, the event acquisition function 34c proceeds to step S206 again to acquire the event with the event name set in step S203 or modified in step S205 for the newly acquired X-ray image. That is, the event acquisition function 34c can acquire events based on X-ray images.

[0087] Alternatively, the event acquisition function 34c can also automatically determine whether the event has ended in step S207 by comparing the location information of the surrounding devices with the database. Furthermore, the event acquisition function 34c can also determine whether the event has ended by accepting input from the user via the input interface 31.

[0088] On the other hand, if the event ends (step S207 affirmative), the event acquisition function 34c determines whether there is a next event (step S208). If there is a next event (step S208 affirmative), the event acquisition function 34c returns to step S203 to set an event name for the next event. On the other hand, if there is no next event (step S208 negative), the event acquisition function 34c ends the process.

[0089] That is, according to Figure 3BThe processing described above involves image acquisition function 34b sequentially acquiring X-ray images during the treatment procedure on the subject P, and event acquisition function 34c sequentially acquiring events during the treatment procedure. For example, event acquisition function 34c determines, based on the X-ray images, whether an event with an event name set in step S203 or modified in step S205 has ended, thereby enabling the sequential acquisition of events for newly acquired X-ray images. Management function 34d can manage X-ray images and events by establishing a correlation with time information during the treatment procedure.

[0090] Next, regarding Figure 3C Explanation will be provided. In Figure 3C In the illustrated case, the event acquisition function 34c accepts patient information input (step S301) before the start of the treatment procedure, and also accepts the selection of the procedure (step S302). For example, the event acquisition function 34c can execute steps S301 and S302 by accepting input operations from the user via the input interface 31. Alternatively, steps S301 and S302 can be omitted as appropriate.

[0091] Next, the image acquisition function 34b acquires the X-ray image collected in the X-ray diagnostic apparatus 10 (step S303). Next, the event acquisition function 34c sets an event name by processing the X-ray image acquired in step S303 (step S304). For example, the event acquisition function 34c detects a medical device from the X-ray image acquired in step S303, determining the type of the detected medical device and the location of the subject P within the body. For example, if a balloon used for dilation of a stenotic vascular segment is detected as a medical device, and the balloon is detected to expand frame by frame based on preceding and following frames, the event acquisition function 34c can acquire the event of "balloon dilation." That is, the event acquisition function 34c can acquire events based on X-ray images. The management function 34d manages the X-ray images and events in association with time information during the treatment procedure.

[0092] Next, the event acquisition function 34c determines whether a next X-ray image exists (step S305). If a next X-ray image exists (step S305 affirmative), the process proceeds to step S303 again. On the other hand, if no next X-ray image exists (step S305 negative), the event acquisition function 34c ends the process. That is, according to... Figure 3C The processing shown includes image acquisition function 34b acquiring X-ray images sequentially during the treatment procedure on the subject P, event acquisition function 34c acquiring events during the treatment procedure based on the X-ray images, and management function 34d managing the X-ray images and events in association with the time information during the treatment procedure.

[0093] also, Figure 3A , Figure 3B and Figure 3C This is just one example; the event acquisition function 34c can acquire events through various methods. For instance, the event acquisition function 34c can also acquire events based on images collected by a medical imaging diagnostic device (medical instrument) different from the X-ray diagnostic device 10. For example, the event acquisition function 34c can also... Figure 3C In step S304, the event name is set based on the ultrasound image collected in parallel with the X-ray image.

[0094] Event retrieval function 34c, for example, Figure 4 As shown, the various event acquisition methods described above can be executed based on a database storing event categories. Furthermore, this database can be stored in memory 33 and retrieved appropriately. For example, the event acquisition function 34c compares various information, such as control information from the holding device and examination table, input information from the input terminal, input information from the microphone or camera, and ultrasound images collected by IVUS (intravascular ultrasound), with the event names registered in the database to obtain matching or similar event names. Additionally, Figure 4 The holding device is, for example, Figure 2 The C-shaped arm 105 is shown. Additionally... Figure 4 The examination bed, for example, is for Figure 2 The device shown controls the movement and tilting of the top plate 104. Additionally, Figure 4 The input terminal is Figure 1 One example of the input interface 31 shown is, for example, a mouse or keyboard. Additionally, Figure 4 The microphone and camera are Figure 1 This is an example of input interface 31. Furthermore, IVUS-based ultrasound images can be stored in memory 33 and retrieved appropriately. That is, by pre-registering event names in the database and using those event names to retrieve events, the event retrieval function 34c can retrieve the same event name for the same type of event, thus efficiently managing events. Additionally, Figure 4 This diagram illustrates an example of the event acquisition and processing in the first embodiment.

[0095] Next, the output function outputs the X-ray image in a manner that allows for identification of its relationship to the events. For example, output function 34e causes display 32 to display a timeline of events arranged in a time sequence based on time information that establishes a correlation between the X-ray image and the events. Furthermore, output function 34e causes display 32 to display the X-ray image in a manner that allows for identification of the time sequence relationship between the events displayed on the timeline.

[0096] For example, output function 34e, such as Figure 5 As shown, the display 32 displays a screen including areas R11, R12, and R13. For example, the output function 34e, during a surgical procedure, enables... Figure 5 The display screen is shown on monitor 32. Additionally, Figure 5 This is a diagram showing an example of the first embodiment.

[0097] Specifically, output function 34e displays a timeline of events configured by time sequence in region R11. Figure 5 As an example, the text illustrates a timeline representation of events E11 (arrival at the target site), E12 (L-ICM balloon angiography), and E13 (stent placement). Furthermore, Figure 5 For event E12, it refers to the situation consisting of detailed events such as event E121 (confirmation of the stenotic site (injection of contrast agent)), event E122 (inflation of the balloon), and event E123 (confirmation of the stenotic site (injection of contrast agent)).

[0098] Furthermore, output function 34e displays the current event as line L1 in region R11. That is, in Figure 5 In this context, event E12, which includes event E121, is the current event, event E11 is a past event, and event E13 is the next event. Furthermore, output function 34e can display the X-ray image associated with event E121 in regions R12 and R13. For example, output function 34e will display the X-ray image I11, collected during event E121, as a thumbnail in region R12 and magnified in region R13. That is, by displaying line L1 in the timeline, output function 34e can display the X-ray image I11 in a way that allows identification of the temporal sequence relationship between the events displayed in the timeline. For example, team members in a treatment procedure can refer to... Figure 5 The display allows for understanding the current situation of the treatment surgery and facilitates easy communication.

[0099] Furthermore, output function 34e can display all events included in the treatment procedure, or only a portion of them. For example, output function 34e can also select events to display based on a list of events to be displayed, and events not included in the list are set to not be displayed. Thus, output function 34e can reduce the number of events displayed, avoiding user confusion caused by displaying too much information.

[0100] Furthermore, while line L1 is described as displaying the current event, it can also be moved arbitrarily along the timeline. For example, during a surgical procedure, the user can move line L1 to the position of event E11, which is a past event. In this case, output function 34e causes display 32 to show an X-ray image associated with event E11 where the moved line L1 is located. Thus, when displaying X-ray images collected during the execution of the surgical procedure as reference images, information about the situation during the surgical procedure can be used to narrow down the candidates, thereby reducing the hassle of finding the necessary images.

[0101] Here, output function 34e can also display, in addition to the X-ray image associated with the event where line L1 is located, a related image associated with that X-ray image. For example, as Figure 5 As shown, when online L1 is located at event E121, output function 34e, in addition to displaying the X-ray image I11 associated with event E121, can also display associated images such as I12, I13, and I14 in area R12. Furthermore, if the user selects any of these associated images, output function 34e can also magnify and display the associated image in area R13.

[0102] Furthermore, the associated image of X-ray image I11 is, for example, an image with the same or similar ancillary information as X-ray image I11. For example, output function 34e retrieves a medical image similar to X-ray image I11 with ancillary information such as patient information and events from medical images stored in memory 33 or image storage device 20, and displays it as an associated image on display 32. As an example, output function 34e displays the X-ray image collected from subject P in event E11, which is a past event, as an associated image of X-ray image I11 on display 32.

[0103] Additionally, for example, the associated image of X-ray image I11 is an image with similar characteristics to X-ray image I11. As an example, output function 34e retrieves images from medical images stored in memory 33 or image storage device 20 that appear on medical devices identical or similar to X-ray image I11, images collected for the same organs as X-ray image I11, etc., and displays them as associated images on display 32.

[0104] Additionally, for example, the associated image of X-ray image I11 is an X-ray image with the same or similar arm position information as X-ray image I11. That is, output function 34e retrieves X-ray images with the same or similar imaging position and angle as X-ray image I11 from medical images stored in memory 33 or image storage device 20, and displays them as associated images on display 32. Thus, for example, when displayed during surgical procedures, output function 34e can shrink the displayed associated image to near the real-time arm angle.

[0105] Furthermore, when selecting associated images based on arm position information, X-ray images that are dissimilar to the arm position information are excluded from the associated images. Here, output function 34e can also notify the user that there are X-ray images omitted from display due to dissimilar arm position information. For example, output function 34e can also display text on the timeline indicating that there are X-ray images omitted from display due to dissimilar arm position information.

[0106] Furthermore, the associated image with X-ray image I11 can also be a processed image generated based on X-ray image I11. For example, output function 34e can also retrieve a PI (Parametric Imaging) image generated based on X-ray image I11 from medical images stored in memory 33 or image storage device 20, and display it on display 32 as an associated image with X-ray image I11. Moreover, a PI image refers to a color image generated by calculating parameters such as blood flow arrival time from multiple frames of X-ray images obtained through angiography of blood vessels, and assigning colors corresponding to the parameter values ​​to each pixel.

[0107] Here, output function 34e can read both the generated processed image and the application used to create the processed image. For example, output function 34e can read both the PI image stored in memory 33 and image storage device 20, and the application used to generate the PI image. In the case of reading the application, output function 34e can, for example, generate a PI image from an X-ray image containing multiple frames of X-ray image I11, and display it as a correlated image on display 32.

[0108] Furthermore, the output function 34e can also process or generate associated images based on the X-ray image I11 and display them. For example, when a PI image based on the X-ray image I11 is read out, the output function 34e can process the display mode of the PI image, such as WL (Window Level) and WW (Window Width), in a way that is easy to compare with the X-ray image I11, and then display it as an associated image. Alternatively, for example, when an application used to create a PI image is read out, the output function 34e can set the WL and WW in a way that is easy to compare with the X-ray image I11, generate a PI image, and display it as an associated image.

[0109] Furthermore, while a PI image has been described as an example of a processed image generated based on X-ray image I11, the implementation is not limited to this. For example, output function 34e can also display a vascular mural image obtained by extracting only the vessel wall from X-ray image I11, or a simplified image with guide lines marked on X-ray image I11, as an associated image associated with X-ray image I11 on display 32. Additionally, output function 34e can also display a vascular mural image generated based on X-ray images collected prior to X-ray image I11, or an overlay image of a simplified image and X-ray image I11, as an associated image associated with X-ray image I11 on display 32.

[0110] Here, output function 34e can also display the processed image near the original image. For example, output function 34e can display the processed image generated based on X-ray image I11 at a position on the timeline adjacent to X-ray image I11. Alternatively, output function 34e can also display the processed image at a position on the timeline corresponding to the date and time of its creation.

[0111] Furthermore, the associated image with X-ray image I11 is not limited to X-ray images; it can also be an image collected using a device different from the X-ray diagnostic apparatus 10. For example, output function 34e can also display an ultrasound image collected during event E121 as an associated image with X-ray image I11 on display 32. Additionally, output function 34e can also display ultrasound images previously collected from subject P as associated images on display 32. As an example, output function 34e can also display IVUS images or TEE (Transesophageal Echocardiography) images collected from subject P as associated images on display 32. In addition, the output function 34e can also display various medical images, such as X-ray CT (Computed Tomography) images, MRI (Magnetic Resonance Imaging) images, SPECT (Single Photon Emission: Computed Tomography) images, PET (Positron Emission: Computed Tomography) images, and OCT (Optical Coherence Tomography) images, as associated images on the display 32.

[0112] Furthermore, the associated image with X-ray image I11 can also be a two-dimensional medical image generated from a three-dimensional medical image. For example, output function 34e can also generate a two-dimensional medical image by rendering three-dimensional medical images such as preoperative CT or CBCT (Cone-Beam Computed Tomography) images, and display it as an associated image on display 32. Here, output function 34e can also generate a two-dimensional medical image based on arm position information attached to X-ray image I11. That is, output function 34e can also generate a two-dimensional medical image whose imaging position and angle are similar to those of X-ray image I11, and display it as an associated image with X-ray image I11 on display 32. Thus, output function 34e can, for example, shrink the displayed associated image to near the real-time arm angle when displaying it during surgical procedures.

[0113] Furthermore, as described above, when there are multiple types of associated images, the output function 34e can also display a collapsed associated image based on user instructions. For example, the output function 34e can also be as follows: Figure 5As shown, icons B11, B12, etc., are displayed. Operations on these icons cause the displayed associated image to shrink. For example, icon B11 corresponds to "ultrasound image". Here, when the user operates on icon B11, output function 34e can toggle whether to display an ultrasound image as an associated image associated with X-ray image I11.

[0114] For example, output function 34e, such as Figure 6 As shown, a database can be used to retrieve associated images. For example, the event acquisition function 34c can perform a database search based on keywords input from the input terminal, accompanying information attached to the collected image such as the X-ray image I11, and image features of the collected image, and display the associated image as a monitor such as the display 32. Furthermore, Figure 6 This diagram illustrates an example of the associated image acquisition process in the first embodiment.

[0115] In addition, Figure 5 The details of events included in event E12 (events E121, E122, and E123) are also displayed and explained, but output function 34e can also be used as follows. Figure 7 As shown, the display of these detailed events is omitted. That is, output function 34e can also limit the hierarchy of events to be displayed. Thus, output function 34e can control the amount of information provided to the user. Furthermore, Figure 7 This is a diagram showing an example of the first embodiment.

[0116] The following explains the hierarchical display of events. First, the management function 34d, for example... Figure 8 As shown, this allows the events acquired by the event retrieval function 34c to be hierarchically managed. Figure 8 In the scenario shown, the treatment procedure includes four major events: "Surgery Start," "Animal Angiography," "Stent Placement," and "Surgery End." Additionally, Figure 8 This is a diagram illustrating the hierarchical display of events in the first embodiment.

[0117] Here, the "Surgery Start" event is associated with images I21, I22, I23, and I24. Furthermore, "Surgery Start" includes "Guidewire Insertion" as a more detailed event. Here, the "Guidewire Insertion" event is associated with images I22 and I24, which are associated with the "Surgery Start" event. Additionally, "Guidewire Insertion" includes "Branch Confirmation" as a more detailed event. Here, the "Branch Confirmation" event is associated with image I22, which is associated with the "Guidewire Insertion" event. Thus, higher-level events (major events) are associated with more images, while lower-level events (detailed events) have fewer associated images. Similarly, "Affected Area Angiography" includes "Affected Area Angiography" as a more detailed event. Furthermore, "Stent Procedure" includes "Stent Placement" as a more detailed event.

[0118] In addition, "stent placement" includes "stent alignment", "dilation procedure" and "post-dilation confirmation" as more detailed events.

[0119] Output function 34e is based on such as Figure 8 Events are managed hierarchically as shown, and their display is controlled accordingly. For example, output function 34e... Figure 9A and Figure 9B As shown, this displays the event hierarchy corresponding to the input actions received from the user. Additionally, Figure 9A and Figure 9B This is a diagram showing an example of the first embodiment.

[0120] Specifically, output function 34e firstly... Figure 9A As shown, the output function 34e displays four main events: "Surgery Start," "Affected Area Angiography," "Stent Placement," and "Surgery End." For example, if the user selects "Stent Placement," the output function 34e further displays the detailed event within "Stent Placement," namely "Stent Indwelling." Thus, the output function 34e avoids providing the user with excessive information while providing detailed events according to the user's needs.

[0121] To give another example, output function 34e, such as Figure 10A and Figure 10B As shown, events are displayed at the specified hierarchy. Additionally, Figure 10A and Figure 10B This is a diagram showing an example of the first embodiment.

[0122] For example, if "4 layers" are set as a prescribed hierarchy, output function 34e is as follows: Figure 10AAs shown, events such as "Surgery Start," "Affected Area Angiography," "Stent Procedure," and "Surgery End" in the first layer, "Guidewire Insertion," "Affected Area Angiography," "Stent Placement," and "Post-Dilation Confirmation" in the second layer, "Branch Confirmation," "Stent Position," and "Dilation Implementation" in the third layer, and "Angiography Confirmation" in the fourth layer are displayed separately. On the other hand, when "2 layers" are set as the defined hierarchy, output function 34e... Figure 10B As shown, events such as "surgery start," "anechoic angiography of the affected area," "stent placement," and "surgery end" in the first layer, and events such as "guidewire insertion," "anechoic angiography of the affected area," "stent placement," and "post-dilation confirmation" in the second layer are displayed separately. That is, when "2 layers" are set as the prescribed level, output function 34e omits the display of the third layer and beyond. Therefore, output function 34e avoids confusion caused by overly detailed events. Furthermore, the prescribed level can be a preset value, a value automatically set by output function 34e based on the total number of events, or a value arbitrarily set by the user.

[0123] As described above, according to the first embodiment, the image acquisition function 34b sequentially acquires medical images during the treatment procedure on the subject P. Furthermore, the event acquisition function 34c, based on the medical images, sequentially acquires events during the treatment procedure. Additionally, the management function 34d manages the medical images and events in association with time information during the treatment procedure. Furthermore, the output function 34e outputs the medical images in a manner that allows for identification of their relationship to the events. Therefore, the medical information processing apparatus 30 of the first embodiment can utilize medical images more effectively. For example, during the treatment procedure, team members can easily communicate their understanding of the current situation. Furthermore, for example, during the execution of the treatment procedure, when it is desired to use the medical images collected during the treatment procedure, the necessary images can be easily located.

[0124] Furthermore, as described above, according to the first embodiment, the output function 34e can further display associated images. Therefore, for example, when medical images are desired during the execution of a surgical procedure, the necessary images can be more easily located.

[0125] Furthermore, as described above, according to the first embodiment, the management function 34d manages events by hierarchizing them. Additionally, the output function 34e displays events at the level corresponding to the input operation received from the user, or displays events at a predetermined level. Thus, the medical information processing device 30 can avoid user confusion caused by displaying too much information.

[0126] Furthermore, while the description assumes that medical images are displayed on the display 32 in a manner that allows for the identification of their relationship to events, the implementation is not limited to this. For example, output function 34e could also be made. Figure 5 or Figure 7 The displayed image is sent to the X-ray diagnostic apparatus 10. In this case, the X-ray diagnostic apparatus 100 can display the image on its monitor 108. Figure 5 , Figure 7 The display shown is shown below.

[0127] (Second Implementation)

[0128] In the first embodiment described above, as an example of display, for Figure 5 , Figure 7 The display screen has been described. In contrast, in the second embodiment, as a display example, the screen shown is... Figure 11 The display screen will be explained. Figure 11 This is a diagram illustrating a display example of the second embodiment. The medical information processing system 1 of the second embodiment has... Figure 1 and Figure 2 The medical information processing system 1 shown has the same structure, but the processing part of the output function 34e is different. Hereinafter, points having the same structure as those described in the first embodiment will be labeled as... Figure 1 and Figure 2 The same reference numerals are used in the accompanying drawings, and the descriptions are omitted.

[0129] For example, output function 34e, such as Figure 11 As shown, the display 32 displays a screen including regions R21 and R22. Specifically, the output function 34e displays a timeline of events arranged in a time sequence in region R21, and displays medical images obtained during the treatment procedure on the timeline. That is, the output function 34e displays a timeline of events arranged in a time sequence, and displays medical images in a manner that allows identification of the time sequence relationship between the events displayed on the timeline.

[0130] Specifically, in Figure 11 In the case shown, the treatment procedure includes event E21, "reaching the target site," and event E22, "L-ICM balloon angiography." Furthermore, in event E21, X-ray images I21, I22, and I23 are collected sequentially; in event E22, X-ray images I24 and I25 are collected sequentially. Additionally, with... Figure 5 and Figure 7Similarly, output function 34e can display related images such as ultrasound images in addition to the X-ray images collected during the treatment procedure. Furthermore, output function 34e can also display icons B21, B22, etc., and the displayed related images can be shrunk by manipulating these icons.

[0131] Furthermore, the output function 34e, such as Figure 11 As shown, the arm position information during X-ray image collection can be displayed. Specifically, the output function 34e can display X-ray images collected at "LAO45°, CAU35°" I21, "LAO60°, CAU35°" I22, "LAO60°, CAU30°" I23, "LAO45°, CAU35°" I24, and "LAO60°, CAU35°" I25.

[0132] Therefore, for example, image reproduction can be performed efficiently. For instance, during surgical procedures, the user refers to... Figure 11 The display screen determines that, in the current situation, it is preferable to use an image with the same imaging angle as X-ray image I21, and rotates the C-arm 105 in a manner that becomes "LAO45°, CAU35°". This allows the user to perform treatment surgery more efficiently using real-time images at the desired angle.

[0133] Additionally, the output function 34e can also be used in... Figure 11 In region R21, the arm position information most frequently used during the treatment procedure will be highlighted. That is, output function 34e can also estimate and display the main working angle. Such working angles are mostly the angles from which useful images are obtained during the treatment procedure, and there are many situations where image reproduction is required. Therefore, by highlighting the arm position information most frequently used during the treatment procedure, image reproduction can be performed more efficiently.

[0134] (Third Implementation)

[0135] In the first to second embodiments described above, as an example of display, for Figure 5 , Figure 7 , Figure 11 The display screen was described. In contrast, in the third embodiment, as a display example, the screen shown was... Figure 12 The display screen will be explained. Figure 12 This is a diagram illustrating a display example of the third embodiment. The medical information processing system 1 of the third embodiment has... Figure 1 and Figure 2The medical information processing system 1 shown has the same structure, but the processing part of the output function 34e is different. Hereinafter, points having the same structure as those described in the first to second embodiments will be labeled as... Figure 1 and Figure 2 The same reference numerals are used in the accompanying drawings, and the descriptions are omitted.

[0136] For example, output function 34e, such as Figure 12 As shown, the display 32 displays a screen including regions R31, R32, and R33. Specifically, the output function 34e displays a timeline in region R31 that arranges events in a time sequence. Additionally, the output function 34e displays medical images obtained during surgical procedures in region R32. Here, when displaying X-ray images as medical images, the output function 34e can also be used as follows... Figure 12 As shown, the arm position information during the collection of each X-ray image is displayed.

[0137] Additionally, output function 34e displays information about input and output between the user and the user in area R33. For example, the user inputs "request angiography image" via voice input, keyboard operation, etc. In response, output function 34e retrieves an X-ray image obtained by angiography of the blood vessels of the subject P from X-ray images collected during the treatment procedure and displays it in area R33. Thus, output function 34e displays medical images based on input operations received from the user, and before displaying the medical images, it causes area R33 to display information about input and output between the user and the user.

[0138] That is, output function 34e is an AI (Artificial Intelligence) that displays medical images based on user input, and enables the exchange between the user and the AI ​​to be displayed in area R33. This allows the user to more easily identify the necessary images. For example, by displaying the medical image along with the instructions entered to access the medical image in area R33, the user can clearly understand the intent behind the accessed medical image.

[0139] (Fourth Implementation)

[0140] In the first to third embodiments described above, as an example of display, for Figure 5 , Figure 7 , Figure 11 , Figure 12 The display screen was described. In contrast, in the fourth embodiment, as a display example, the screen shown... Figure 13 The display screen will be explained. Figure 13 This is a diagram illustrating a display example of the fourth embodiment. The medical information processing system 1 of the fourth embodiment has... Figure 1 and Figure 2 The medical information processing system 1 shown has the same structure, but the processing part of the output function 34e is different. Hereinafter, points having the same structure as those described in the first to second embodiments will be labeled as... Figure 1 and Figure 2 The same reference numerals are used in the accompanying drawings, and the descriptions are omitted.

[0141] For example, output function 34e, such as Figure 13 As shown, a timeline is displayed that arranges events such as "angiography," "balloon," and "stent" in a time sequence, and multiple medical images I41 are displayed in a way that allows identification of the time sequence relationship between these events. Furthermore, the output function 34e displays supplementary information for the medical images. For example, the output function 34e displays supplementary information such as "angle," "AutoPos," and "Vital" for each medical image. Additionally, this supplementary information can be pre-stored in the DICOM tags of each medical image, for example. For example, by referring to this supplementary information, the user can more easily find the image they want to confirm.

[0142] Additionally, the output function 34e can further display images of the examination room. For example, such as... Figure 13 As shown, output function 34e can also display multiple examination room images I42. Specifically, during the treatment procedure, image acquisition function 34b sequentially acquires examination room images obtained by photographing the examination room during the treatment procedure. These examination room images are, for example, multiple frames contained in a moving image captured using an optical camera. In addition, management function 34d manages the examination room images in association with time information, in addition to medical images and events.

[0143] Additionally, the output function 34e, such as Figure 13 As shown, a timeline is displayed, showing events such as "angiography," "balloon," and "stent" arranged in a time sequence. Furthermore, output function 34e displays multiple medical images I41 in a manner that identifies the time sequence relationships between the events displayed in the timeline. Also, output function 34e displays examination room images I42 in association with the timeline. That is, output function 34e displays each event of "angiography," "balloon," and "stent," multiple medical images I41, and multiple examination room images I42 in a horizontal axis (time sequence) matching manner. For example, by referring to the examination room images during surgery, the user can grasp the overall picture of the surgery, making the medical images contribute to the efficiency of the procedure.

[0144] (Fifth Implementation)

[0145] Furthermore, the first to fourth embodiments have been described, but various other implementation methods may be used in addition to the embodiments described above.

[0146] For example, in Figure 5 , Figure 7 , Figure 11 , Figure 12 , Figure 13 In the examples shown, the case where at least a timeline arranging events in chronological order is displayed is illustrated. However, the implementation is not limited to this. For example, output function 34e may omit the display of the timeline and display medical images associated with events received by the user via sound or keyboard input.

[0147] Furthermore, for example, in the above embodiment, the processing circuit 34 of the medical information processing device 30 is described as performing functions such as event acquisition function 34c, management function 34d, and output function 34e. However, the embodiment is not limited to this; for example, the processing circuit 110 of the X-ray diagnostic device 10 may perform functions equivalent to event acquisition function 34c, management function 34d, and output function 34e.

[0148] For example, the processing circuit 110 also performs an event acquisition function 110d (not shown) and a management function 110e. For example, during a treatment procedure on a subject P, the acquisition function 110b sequentially acquires X-ray images based on X-rays transmitted through the subject P. Additionally, the event acquisition function 110d sequentially acquires events during the treatment procedure based on the X-ray images. Furthermore, the management function 110e manages the X-ray images and events in association with time information during the treatment procedure. Then, the output function 110c outputs the X-ray images in a manner that allows identification of their relationship to the events. For example, the output function 110c enables the display 108 to show... Figure 5 , Figure 7 , Figure 11 , Figure 12 , Figure 13 The display screen for, etc.

[0149] Here, the collection function 110b can also perform X-ray image collection again based on the collection conditions of the X-ray image selected by the user in the X-ray image output by the output function 110c. For example, when the output function 110c makes... Figure 11When the display screen is shown on monitor 108 and the user selects X-ray image I21, the collection function 110b performs X-ray image collection again under the condition of "LAO 45°, CAU 35°". For example, if the designated action button is pressed while X-ray image I21 is displayed in area R22, the collection function 110b will collect the selected X-ray image under the condition of "LAO 45°, CAU 35°". This allows for more efficient image reproduction.

[0150] The term "processor" used in the above description refers to circuits such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), Application Specific Integrated Circuit (ASIC), and programmable logic devices (e.g., Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), and Field Programmable Gate Array (FPGA)). When the processor is, for example, a CPU, the processor implements its function by reading and executing a program stored in a memory circuit. Alternatively, when the processor is, for example, an ASIC, the function is directly loaded as logic circuitry into the processor's circuitry instead of storing the program in a memory circuit. Furthermore, the processors in this embodiment are not limited to being configured as a single circuit for each processor; multiple independent circuits can be combined to form a single processor to implement its function. Moreover, multiple components shown in the figures can be integrated into a single processor to implement the function.

[0151] The constituent elements of the devices in the above-described embodiments are functional concepts and do not necessarily need to be physically configured as shown in the illustrations. That is, the specific methods of distributing or merging the devices are not limited to those shown in the illustrations, and they can be functionally or physically distributed or merged in any unit according to various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed in each device can be implemented by a CPU and the program parsed and executed by the CPU, or they can be implemented as hardware based on wiring logic.

[0152] Furthermore, the medical information processing method described in the above embodiments can be implemented by executing a pre-prepared medical information processing program on a computer such as a personal computer or workstation. This medical information processing program can be published via a network such as the Internet. Alternatively, the medical information processing program can be recorded on a computer-readable non-transitory recording medium such as a hard disk, floppy disk (FD), CD-ROM, MO, or DVD, and executed by a computer from the recording medium.

[0153] According to at least one of the embodiments described above, medical images can be utilized more effectively.

[0154] Several embodiments have been described, but these embodiments are given by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other ways, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope or spirit of the invention, as well as in the scope of the invention as described in the claims and its equivalents.

Claims

1. A medical information processing device, comprising: The image acquisition unit sequentially acquires X-ray or ultrasound images during the treatment procedure on the subject. The determining unit, based on the sequentially acquired X-ray images or ultrasound images, sequentially determines images in the sequentially acquired X-ray images or ultrasound images in which an event representing the situation of the treatment surgery has occurred; and The management department stores the acquired X-ray images or ultrasound images, and stores information about the event that occurred in association with the images, wherein the stored X-ray images or ultrasound images are the ones from which the event was determined to have occurred.

2. The medical information processing device according to claim 1, wherein, It also includes an output section that displays the X-ray or ultrasound images stored in the management section in a manner that allows for the identification of the chronological relationship between the information and the events that occurred.

3. The medical information processing device according to claim 2, wherein, The output unit displays a timeline that arranges the events in a time sequence based on time information, and displays the X-ray image or the ultrasound image on the display unit in a manner that allows identification of the time sequence relationship between the events displayed in the timeline.

4. The medical information processing device according to claim 3, wherein, The output unit also displays associated images linked to the X-ray image or the ultrasound image.

5. The medical information processing device according to claim 4, wherein, The associated image is an image that has the same accompanying information as the X-ray image or the ultrasound image, or an image whose features are the same as the X-ray image or the ultrasound image.

6. The medical information processing device according to claim 4 or 5, wherein, The associated image is an image collected using a different device than the X-ray image or the ultrasound image.

7. The medical information processing device according to claim 4, wherein, The associated image is a processed image generated based on the X-ray image or the ultrasound image.

8. The medical information processing device according to claim 4, wherein, The X-ray images are X-ray images collected using an X-ray diagnostic device. The output unit displays the X-ray image in a manner that allows identification of its relationship to the event, and displays an X-ray image with the same arm position information as the associated image.

9. The medical information processing device according to claim 4, wherein, The X-ray images are X-ray images collected using an X-ray diagnostic device. The output unit displays the X-ray image in a manner that allows identification of its relationship to the event, and displays a two-dimensional X-ray image or ultrasound image generated from a three-dimensional X-ray image or ultrasound image based on the arm position information of the X-ray image as the associated image.

10. The medical information processing device according to claim 4, wherein, The output unit processes or generates the associated image based on the X-ray image or the ultrasound image, and then displays the associated image.

11. The medical information processing device according to claim 3, wherein, The X-ray images are X-ray images collected using an X-ray diagnostic device. The output unit displays the X-ray image in a manner that allows identification of its relationship to the event, and displays the arm position information at the time of X-ray image collection.

12. The medical information processing device according to claim 3, wherein, The output unit displays the X-ray image or the ultrasound image based on the input operation received from the user, and displays the input and output information between the user and the user before displaying the X-ray image or the ultrasound image.

13. The medical information processing device according to claim 3, wherein, The image acquisition unit also sequentially acquires images of the examination room obtained by taking photographs during the treatment procedure. The management department manages the X-ray or ultrasound images, the events, and the images in the examination room by associating them with the time information. The output unit displays a timeline that arranges the events in a time sequence based on the time information, and displays the X-ray image or the ultrasound image in a manner that can identify the time sequence relationship between the events displayed in the timeline, thereby enabling the images in the examination room to be displayed in association with the timeline.

14. The medical information processing device according to claim 13, wherein, The output unit displays the X-ray image or the ultrasound image together with accompanying information of the X-ray image or the ultrasound image.

15. The medical information processing device according to any one of claims 1 to 5, 7 to 14, wherein, The determining unit retrieves events from the treatment procedure based on a database storing event categories.

16. The medical information processing device according to claim 3, wherein, The management department manages the events by creating a hierarchical structure. The output unit displays the event at the level corresponding to the input operation received from the user.

17. The medical information processing device according to claim 3, wherein, The management department manages the events by creating a hierarchical structure. The output unit displays the events at the specified level.

18. An X-ray diagnostic device, comprising: The collection unit sequentially collects X-ray images based on the X-rays transmitted through the subject during the treatment surgery. The event acquisition unit acquires events representing the situation of the treatment surgery sequentially during the treatment surgery based on the X-ray image; The management department will manage the X-ray images and events in association with the time information during the treatment procedure. as well as The output unit outputs the X-ray image in a manner that allows for the identification of its relationship with the event.

19. The X-ray diagnostic apparatus according to claim 18, wherein, The collection unit performs X-ray image collection again based on the collection conditions of the X-ray image selected by the user in the X-ray image output by the output unit.

20. A storage medium non-volatilely storing a medical information processing program for causing a computer to perform the following processes: X-ray or ultrasound images are obtained sequentially during the treatment procedure on the subject. Based on the X-ray image or the ultrasound image, events representing the state of the treatment procedure are sequentially acquired during the treatment procedure. The X-ray or ultrasound images and the events are linked to time information during the treatment procedure for management purposes. The X-ray image or the ultrasound image is output in a manner that allows for identification of its relationship to the event.

Images