Medical information processing device, medical diagnostic device, and medical information processing method

Abstract

To efficiently perform support to a user on an operation of a medical device.SOLUTION: A medical information processing device includes an acquisition unit, an answer acquisition unit, a determination unit, and a display control unit. The acquisition unit acquires a user input on an operation of a medical device. The answer acquisition unit inputs the acquired user input to a learned model that has learned to output an answer to the user input when receiving the input of the user input on the operation of the medical device, and acquires output of the learned model. The determination unit determines execution possibility by a self-device on the acquired output. The display control unit causes a display unit to display information corresponding to the output on the basis of the execution possibility.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The embodiments disclosed in the present specification and drawings relate to a medical information processing apparatus, a medical diagnostic apparatus, and a medical information processing method. [Background technology]

[0002] Traditionally, support for users regarding the operation of medical diagnostic equipment such as X-ray diagnostic equipment has been provided by providing operation manuals. When users do not know how to operate a medical diagnostic equipment, they read the manual to search for relevant sections or solutions. However, this support method is inefficient, as it takes time to find the operation the user wants in the manual. Also, depending on the status of the equipment, multiple operations may need to be performed in the appropriate order. However, even reading the manual may not lead to the answer. It is also difficult for users to determine the appropriate order. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-121837 Summary of the Invention [Problem to be solved by the invention]

[0004] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to efficiently provide support to users regarding the operation of a medical device. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0005] A medical information processing device according to an embodiment includes an acquisition unit, an answer acquisition unit, a determination unit, and a display control unit. The acquisition unit acquires user input related to the operation of a medical device. The answer acquisition unit inputs the acquired user input into a trained model that has been trained to output an answer to the user input when receiving the user input related to the operation of the medical device, and acquires the output of the trained model. The determination unit determines the feasibility of the acquired output on the device itself. The display control unit displays information corresponding to the output on a display according to the feasibility. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an X-ray diagnostic apparatus according to the first embodiment. [Figure 2] FIG. 2 is a diagram showing an example of the appearance of the X-ray diagnostic apparatus according to the first embodiment. [Figure 3] FIG. 3 is a flowchart illustrating a processing procedure of operation support processing by the X-ray diagnostic apparatus according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a display screen displayed on a display in the operation support process according to the first embodiment. [Figure 5] FIG. 5 is a diagram showing an example of a state in which a question is input on the display screen of FIG. [Figure 6] FIG. 6 is a flowchart illustrating the processing procedure of the determination processing by the X-ray diagnostic apparatus according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing an example of how the answers are displayed on the display screen of FIG. [Figure 8] FIG. 8 is a diagram showing an example of a solution displayed on the display screen of FIG. [Figure 9] FIG. 9 is a diagram showing an example of a display screen displayed on a display in the operation support process according to the modified example. DETAILED DESCRIPTION OF THE INVENTION

[0007] Hereinafter, an embodiment of a medical diagnostic apparatus including a medical information processing apparatus will be described in detail with reference to the drawings. In the following description, components having substantially the same functions and configurations are designated by the same reference numerals, and redundant description will be given only when necessary.

[0008] Examples of medical diagnostic devices include X-ray diagnostic devices, X-ray computed tomography (CT) devices, magnetic resonance imaging (MRI) devices, ultrasound diagnostic devices, and PET (positron emission tomography) devices. Examples of X-ray diagnostic devices include X-ray TV bed devices. Medical diagnostic devices may also be called medical image diagnostic devices. Examples of medical diagnostic devices may include single-modality devices such as X-ray diagnostic devices, or multi-modality devices such as PET (positron emission tomography) / CT devices and SPECT (single photon emission CT) / CT devices. In the following embodiments, a medical information processing device mounted on an X-ray diagnostic device will be described as an example, but the medical information processing device may also be mounted on other medical diagnostic devices.

[0009] (First embodiment) Fig. 1 is a block diagram showing an example of the configuration of an X-ray diagnostic apparatus 1 according to the first embodiment. Fig. 2 is a diagram showing an example of the appearance of the X-ray diagnostic apparatus 1 according to the first embodiment. The X-ray diagnostic apparatus 1 is a medical diagnostic apparatus including a medical information processing apparatus 30 according to this embodiment. The X-ray diagnostic apparatus 1 of this embodiment is an X-ray TV bed device used for gastrointestinal contrast examinations and the like.

[0010] As shown in Figures 1 and 2, the X-ray diagnostic device 1 has an X-ray tube 3, an irradiation range limiter 5, an X-ray detector 7, a moving support mechanism 9, a rotating support mechanism 11, a drive unit 13, an output unit 16, and a console device 20.

[0011] The fluoroscopic imaging table has a tabletop 14 that supports the subject, a movement support mechanism 9, an X-ray detector 7, and a rotation support mechanism 11. The fluoroscopic imaging table performs functions such as raising and lowering the tabletop 14, and moving the imaging system (X-ray tube 3, irradiation range limiter 5, and X-ray detector 7) up and down, tilting, and compressing. The X-ray detector 7 detects X-rays that have passed through the subject and transfers them to a console device 20.

[0012] The X-ray tube 3 is connected to a high-voltage generator (not shown). The high-voltage generator generates a tube current to be supplied to the X-ray tube 3 and a tube voltage to be applied to the X-ray tube. The high-voltage generator supplies a tube current suitable for each of X-ray imaging and X-ray fluoroscopy to the X-ray tube 3, and applies a tube voltage suitable for each of X-ray imaging and X-ray fluoroscopy to the X-ray tube 3. Specifically, the high-voltage generator generates a tube voltage and a tube current according to the X-ray imaging conditions under the control of the processing circuit 25 using a control function 251 (described later).

[0013] The X-ray tube 3 generates X-rays from an X-ray focus (hereinafter referred to as a tube focus) based on a tube current supplied from a high-voltage generator and a tube voltage applied by the high-voltage generator. The generated X-rays are emitted from an X-ray emission window in the X-ray tube 3. For example, when performing a gastrointestinal contrast agent examination, the X-ray tube 3 irradiates X-rays onto a subject P who has been administered a contrast agent and a foaming agent.

[0014] The irradiation range limiter 5 is provided in front of the X-ray tube 3 and between the X-ray tube 3 and the X-ray detector 7. Specifically, the irradiation range limiter 5 is provided in front of the X-ray emission window of the X-ray tube 3. The irradiation range limiter 5 is also called an X-ray adjustable diaphragm. The irradiation range limiter 5 limits the X-ray irradiation range so that the X-rays generated by the X-ray tube 3 are not irradiated onto areas other than the region desired by the user to be imaged. For example, the irradiation range limiter 5 limits the irradiation range by moving diaphragm blades in accordance with an instruction to limit the irradiation range input via the input interface 22.

[0015] The irradiation range limiter 5 has a plurality of aperture blades. Each of the plurality of aperture blades is made of lead to block the X-rays generated by the X-ray tube 3. The irradiation range limiter 5 may also have a plurality of filters (hereinafter referred to as additional filters) inserted into the X-ray irradiation field for the purpose of reducing the radiation dose to the subject P and improving image quality. The additional filters are also called X-ray filters, filtering plates, beam filters, radiation quality filters, or beam spectrogram filters.

[0016] The X-ray detector 7 detects the X-rays generated from the X-ray tube 3 and transmitted through the subject P. The X-ray detector 7 is, for example, a flat panel detector (FPD). The X-ray detector 7 has multiple semiconductor detection elements. The semiconductor detection elements convert incident X-rays into electrical signals by direct conversion or indirect conversion. In direct conversion, incident X-rays are directly converted into electrical signals. In indirect conversion, incident X-rays are converted into light by a phosphor, and the light is then converted into an electrical signal.

[0017] Electrical signals generated by the multiple semiconductor detection elements in response to incidence of X-rays are output to an analog to digital converter (hereinafter referred to as the A / D converter), not shown. The A / D converter converts the electrical signals into digital data. The A / D converter outputs the digital data to a pre-processing unit, not shown. Note that an image intensifier or the like may also be used as the X-ray detector 7.

[0018] The movement support mechanism 9 supports the imaging system (X-ray tube 3, irradiation range limiter 5, and X-ray detector 7) so that it can move along the long axis (X-axis) direction of the tabletop 14 under the control of a control function 251 described below. For example, when an imaging method such as long-length imaging in which the imaging system is moved to image the subject is input via the input interface 22, the movement support mechanism 9 moves the imaging system along the first direction in accordance with the imaging timing of the input imaging method. Note that when it is not necessary to move the imaging system, the movement support mechanism 9 fixes the imaging system without moving it relative to the tabletop 14.

[0019] The rotation support mechanism 11 supports the moving support mechanism 9 and the tabletop 14 so that they can rotate (rise and fall) around the short axis (Y-axis) of the tabletop 14 as the rotation axis. The moving support mechanism 9 includes the X-ray tube 3, the irradiation range limiter 5, and the X-ray detector 7. For example, the rotation support mechanism 11 rotates the imaging system (X-ray tube 3, irradiation range limiter 5, and X-ray detector 7) or the tabletop 14 around the rotation axis in response to a user instruction via the input interface 22. The rotation angle of the tabletop 14 around the rotation axis is defined as 0° when the tabletop 14 is positioned horizontally, and as 90° when the tabletop 14 is rotated to a position parallel to the vertical direction. When the tabletop 14 is positioned horizontally, the longitudinal direction of the tabletop 14 is parallel to the X-axis, the lateral direction of the tabletop 14 is parallel to the Y-axis, and the Z-axis, which is perpendicular to the X-axis and Y-axis, is parallel to the vertical direction. Hereinafter, the position of the tabletop 14 when the rotation angle is 90° will be referred to as the standing position. The subject P supported by the tabletop 14 when the rotation angle is 90° will be in the standing position.

[0020] The moving support mechanism 9 and the rotating support mechanism 11 may support the X-ray tube 3, the irradiation range limiter 5, the X-ray detector 7, and the tabletop 14 so that they can move along the three orthogonal axes (X-axis, Y-axis, and Z-axis) shown in Fig. 1. For example, the moving support mechanism 9 supports the X-ray tube 3, the irradiation range limiter 5, and the X-ray detector 7 so that the distance between the focal point of X-ray generation in the X-ray tube and the X-ray detector 7 (source image distance (SID)) can be changed.

[0021] The driver 13 drives the movement support mechanism 9 and the rotation support mechanism 11 under the control of the control function 251. Specifically, the driver 13 drives the rotation support mechanism 11 in accordance with a control signal from the control function 251, causing the rotation support mechanism 11 to rotate about the rotation axis. As a result, each component, such as the tabletop 14, rotates about the rotation axis. For example, when an instruction to position the tabletop 14 in an upright position is input via the input interface 22, the driver 13 rotates the rotation support mechanism 11 to set the rotation angle of the tabletop 14 to 90°. The driver 13 also drives the movement support mechanism 9 in accordance with a user instruction via the input interface 22, thereby moving the imaging system along the X-axis direction.

[0022] A tabletop driving unit (not shown) moves the tabletop 14 by driving the tabletop 14 under the control of the control function 251. Specifically, the tabletop driving unit slides the tabletop 14 along the X-axis direction and the Y-axis direction based on a control signal from the control function 251.

[0023] The output unit 16 includes a speaker 16a and a display 16b. The output unit 16 is controlled by the console device 20, and outputs instructions to the subject by voice or by display. Note that one of the speaker 16a and the display 16b may be omitted.

[0024] The console device 20 has a memory 21, an input interface 22, a display 23, a communication interface 24, and a processing circuit 25. Data communication between the memory 21, the input interface 22, the display 23, the communication interface 24, and the processing circuit 25 is performed via a bus (BUS).

[0025] Although the console device 20 is described as a single console that executes multiple functions, multiple functions may be executed by separate consoles. For example, the functions of the processing circuit 25 described below may be distributed and installed in different console devices.

[0026] The memory 21 is a storage device such as a ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), or integrated circuit storage device that stores various information. The memory 21 may also be a drive device that reads and writes various information from and to a portable storage medium such as a CD-ROM drive, a DVD drive, or a flash memory. The memory 21 does not necessarily have to be realized by a single storage device. For example, the memory 21 may be realized by multiple storage devices. The memory 21 may also be located in another computer connected to the X-ray diagnostic apparatus 1 via a network.

[0027] The memory 21 stores programs executed by the processing circuitry 25, various data used in processing by the processing circuitry 25, and the like. The various data include, for example, current examination images (photographed images and fluoroscopic images from the current examination) and past examination images (photographed images and fluoroscopic images from past examinations). Hereinafter, photographed images and fluoroscopic images are collectively referred to as examination images and X-ray images. The past examination images are, for example, acquired by the processing circuitry 25 from an image server via a network and stored in the memory 21. The program may be, for example, a program that is installed in advance on a computer via a network or from a non-transitory computer-readable storage medium and causes the computer to realize each function of the processing circuitry 25. Alternatively, the program may be stored in a non-transitory computer-readable storage medium and distributed, read from the non-transitory computer-readable storage medium, and installed in the memory 21. Note that the various data used in this specification are typically digital data. The memory 21 is an example of a storage unit.

[0028] The memory 21 also stores a language generation model 211. The language generation model 211 is a trained model that is trained to accept input of a question regarding the operation of the medical device and output an answer to the question. For example, a language generation AI such as chatGPT (registered trademark) can be used as the language generation model 211. However, a language generation AI other than chatGPT may also be used, and a trained model other than a language generation AI may also be used. The language generation model 211 is trained, for example, using a manual regarding the operation of the medical device. The manual is, for example, an instruction manual or data describing the operating procedures of the X-ray diagnostic device 1. The language generation model 211 may also be trained using rules for the operation sequence of the operation axes of the medical device. In this case, for example, specification data or a program that defines the operation sequence of the operation axes of the X-ray diagnostic device 1 is used for training. The specification data or program may be data that is not provided to the user, or may be data specific to the device.

[0029] The answer output from the language generation model 211 includes multiple solutions. That is, the language generation model 211 is trained to output an answer including multiple solutions in response to an input including one question. The answer output from the language generation model 211 may include, in addition to an appropriate operation method for realizing the user's desire, solutions such as information unrelated to the X-ray diagnostic apparatus 1, information that is not an operation method, an operation method that can only be performed on an X-ray diagnostic apparatus 1 of another manufacturer or model, or an operation method that is insufficient to realizing the user's desire. Furthermore, an appropriate operation method may include multiple operations arranged in an appropriate order.

[0030] The input interface 22 accepts various input operations from a user, converts the accepted input operations into electrical signals, and outputs the electrical signals to the processing circuit 25. The user is a medical professional such as a technician or a doctor, and operates the X-ray diagnostic apparatus 1. The input interface 22 according to this embodiment is connected to input devices such as a microphone, a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touchpad, and a touch panel, which inputs instructions by touching the operation surface. The input devices connected to the input interface may also be input devices provided in another computer connected via a network or the like. The user also inputs questions regarding the operation of the apparatus into the input interface 22. The input interface 22 is an example of an input unit.

[0031] The display 23 displays various information in accordance with instructions from the processing circuitry 25. For example, the display 23 displays an X-ray image generated by an image processing function 252 (described later). The display 23 may also display an input screen for inputting X-ray imaging conditions, X-ray fluoroscopy conditions, SID, etc. For example, the display 23 may display a GUI (Graphical User Interface) for accepting various operations from a user. Any display may be used as the display 23, such as a CRT (Cathode Ray Tube) display, a liquid crystal display, an organic EL display, an LED display, or a plasma display. Note that the X-ray diagnostic apparatus 1 may not include the display 23, and the GUI may be displayed on an external display, or the GUI may be displayed via a projector, etc. The display 23 also displays answers to user questions. The display 23 is an example of a display unit.

[0032] The communication interface 24 is an interface for communicating with, for example, a network or an external storage device (not shown). Data such as X-ray images obtained by the X-ray diagnostic apparatus 1 can be transferred to other devices via the communication interface 24 and the network.

[0033] The processing circuitry 25 controls the overall operation of the X-ray diagnostic apparatus 1 in response to electrical signals of input operations output from the input interface 22. The processing circuitry 25 is a processor that executes a control function 251, an image processing function 252, an input acquisition function 253, a response acquisition function 254, a determination function 255, and a display control function 256 by calling and executing programs in the memory 21. Furthermore, the medical information processing device 30 according to this embodiment is realized by the processing circuitry 25 that executes the input acquisition function 253, the response acquisition function 254, the determination function 255, and the display control function 256.

[0034] Although the above-described functions are realized by a single processing circuit 25, this is not a limitation. For example, a processing circuit may be configured by combining multiple independent processors, and each processor may execute a program to realize each function. Furthermore, each of the above-described functions may be called a control circuit, image processing circuit, question acquisition circuit, answer acquisition circuit, determination circuit, and display control circuit, respectively, or may be implemented as a separate hardware circuit. The above description of the functions executed by the processing circuit 25 also applies to the following embodiments and modifications.

[0035] Although the console device 20 is described as a single console that executes multiple functions, the multiple functions may be executed by separate devices. For example, the functions of the processing circuit 25 may be distributed and installed in different devices.

[0036] The term "processor" used in the above description refers to a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD)), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA). When the processor is a CPU, for example, the processor realizes its function by reading and executing a program stored in a memory circuit. On the other hand, when the processor is an ASIC, for example, the program is not stored in a memory circuit, but the function is directly incorporated into the processor circuit as a logic circuit. Note that each processor in this embodiment is not limited to being configured as a single circuit, but may be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, multiple components in FIG. 1 may be integrated into a single processor to realize its function. The above description of "processor" also applies to the following embodiments and modifications.

[0037] The processing circuitry 25 performs overall control of the X-ray diagnostic apparatus 1 according to this embodiment using a control function 251. Specifically, the processing circuitry 25 reads out a control program stored in the memory 21, loads it on the memory, and controls each unit of the X-ray diagnostic apparatus 1 according to the loaded control program. The processing circuitry 25 that realizes the control function 251 is an example of a system control unit.

[0038] In the control function 251, the processing circuitry 25 controls each part of the X-ray diagnostic apparatus 1 using command signals and various initial setting conditions input by the user via the input interface 22. For example, in the control function 251, the processing circuitry 25 controls the drive unit 13 and the tabletop drive unit using information related to the drive of the movement support mechanism 9 and the rotation support mechanism 11 input via the input interface 22. For example, in the control function 251, the processing circuitry 25 controls the rotation or sliding of the tabletop 14, the movement of the imaging system, etc.

[0039] In addition, in the control function 251, the processing circuitry 25 uses information relating to the driving of the irradiation range limiter 5 input from the input interface 22, for example, to control the irradiation range of X-rays.

[0040] In addition, in the control function 251, the processing circuitry 25 reads information such as various initial setting conditions and controls imaging conditions such as the tube voltage, tube current, and irradiation time of the high voltage generator. The imaging conditions may include the product (mAs) of the tube current and irradiation time.

[0041] The processing circuitry 25 generates an X-ray image of the subject based on the output from the X-ray detector 7 using the image processing function 252. For example, in the image processing function 252, the processing circuitry 25 collects fluoroscopic images and radiographic images based on the output from the X-ray detector 7, performs various image processing for display, and displays the X-ray images of the subject, such as the fluoroscopic images and radiographic images, on the display 23. In the image processing function 252, the processing circuitry 25 also generates a moving image of the fluoroscopic image and a still image of the current X-ray image, and displays them on the display 23. In the image processing function 252, the processing circuitry 25 may also read a previous examination image corresponding to the current X-ray image from the memory 21 and display the current X-ray image and the previous examination image in parallel on the display 23. The processing circuitry 25 that realizes the image processing function 252 is an example of an image generation unit and an image processing unit.

[0042] The processing circuitry 25 acquires a question regarding the operation of the medical device using the input acquisition function 253. At this time, the processing circuitry 25 acquires a question input by a user using the input interface 22. The question is, for example, a question sentence input to the language generation model 211 to inquire about the operation of the X-ray diagnostic device 1. The question may consist of only words, or may consist of multiple sentences. Note that the question is not limited to a question, and a command regarding the operation of the X-ray diagnostic device 1 may also be acquired. The command is, for example, a command sentence such as "output a method for changing the device to a standing position." The processing circuitry 25 realizing the input acquisition function 253 is an example of an acquisition unit that acquires a user input regarding the operation of the medical device.

[0043] The processing circuitry 25 inputs the acquired question to the language generation model 211 using the answer acquisition function 254, causes the language generation model 211 to output an answer to the question, and acquires the answer output from the language generation model 211. The answer output from the language generation model 211 includes one or more solutions. The processing circuitry 25 that realizes the answer acquisition function 254 is an example of an answer acquisition unit.

[0044] The processing circuitry 25 uses the determination function 255 to determine the feasibility of each solution output from the language generation model 211 on its own device. The feasibility is an index indicating the degree to which the solution can be implemented on its own device. The own device is the X-ray diagnostic device 1 in which the processing circuitry 25 is installed, and is identified by the purpose, manufacturer name, model name, model number, etc. of the X-ray diagnostic device. Note that an apparatus that matches one or more of the purpose, manufacturer name, model name, model number, etc. of the X-ray diagnostic device may be treated as the own device. For example, the more operations included in the solution that can be executed on the own device, the higher the feasibility. Furthermore, the feasibility may be an index indicating the degree to which a user's desired state will be realized when the solution is executed on the own device. The feasibility may be a numerical value, a classification result into multiple predetermined items, or information indicating whether the solution can be executed on the own device. The processing circuitry 25 realizing the determination function 255 is an example of a determination unit.

[0045] The feasibility of a solution can be determined, for example, by simulating the operation of the device itself. In this case, the processing circuitry 25 simulates the operation of the device itself when operated according to the solution and determines the feasibility based on the simulation results. For example, an interference control program or digital twin technology can be used for the simulation. Alternatively, it may be determined whether the simulation results satisfy predetermined conditions, and solutions that satisfy the conditions may be determined to be highly feasible, while solutions that do not satisfy the conditions may be determined to be less feasible. Alternatively, the simulation results may be used to determine multiple predetermined evaluation items, and the evaluation results may be weighted and added to calculate an evaluation value of the feasibility. Examples of evaluation items that can be used include whether the content is related to a medical device, whether it can be realized by the device itself, whether it is applicable to the current status of the device itself, and whether the user's desired state is achieved.

[0046] The processing circuitry 25 causes the display 23 to display information corresponding to the output of the language generation model 211, depending on the determination result of the feasibility, using the display control function 256. For example, the processing circuitry 25 extracts solutions having a feasibility greater than a predetermined value from the multiple solutions output from the language generation model 211, and displays only the extracted solutions. The processing circuitry 25 may also display the multiple solutions output from the language generation model 211 in descending order of feasibility. The processing circuitry 25 may also process and display the solutions output from the language generation model 211. The processing circuitry 25 that realizes the display control function 256 is an example of a display control unit.

[0047] Next, the operation of the X-ray diagnostic apparatus 1 according to this embodiment will be described. FIG. 3 is a flowchart showing an example of the procedure of operation assistance processing executed by the processing circuitry 25. The operation assistance processing is processing for assisting a user in operation when the user does not know how to operate the X-ray diagnostic apparatus 1 by presenting an appropriate operation method using an answer from a language generation model in response to a question input by the user. FIG. 4 is a diagram showing an example of a display screen (hereinafter referred to as an operation assistance screen) displayed on the display 23 when the operation assistance processing is executed. As shown in FIG. 4, the operation assistance screen displays a question input section 231 and an answer display section 232. A question from the user is input into the question input section 231. The answer display section 232 displays solutions to the question generated using the output of the language generation model 211 according to feasibility.

[0048] The processing procedures described below are merely examples, and each process may be modified as much as possible. Furthermore, steps may be omitted, replaced, or added as appropriate depending on the embodiment.

[0049] (Operation support processing) (Step S101) In the operation support process, the processing circuitry 25 first acquires, as a user input, a question input to the question input unit 231 using the input acquisition function 253, based on an operation signal acquired from the input interface 22. Fig. 5 shows a case where the question "How to put the bed in an upright position" is input to the question input unit 231.

[0050] (Step S102) Next, the processing circuit 25 performs language analysis on the input question using natural language processing with the determination function 255, and estimates the state of the device that the user wants to achieve after the operation. Hereinafter, the state of the device that the user wants to achieve after the operation is referred to as the desired state. The desired state is an example of a target state aimed at by the user and may also be referred to as the final state desired by the user. For example, the desired state is estimated by dividing the question into phrases using natural language processing and converting the words in each phrase into numerical values ​​or symbols using general definitions of words such as "bed" and "standing position." Alternatively, the desired state may be estimated using a conversion rule specific to the device that defines a correspondence between words and numerical values ​​or symbols in advance. Here, it is assumed that in response to the question "How to position the bed in an upright position," the desired state is estimated to be "a state in which the bed is rotated at an angle of 80° or more and 90° or less."

[0051] (Step S103) Next, the processing circuitry 25 generates a prompt to be input to the language generation model 211 based on the input question using the answer acquisition function 254. Specifically, the processing circuitry 25 converts the desired state generated based on the question to create a prompt that gives instructions or commands to the language generation model 211.

[0052] (Step S104) Next, the processing circuitry 25 inputs the created prompt to the language generation model 211 via the answer acquisition function 254, and causes the language generation model 211 to output a solution to the question.

[0053] (Step S105) Next, the processing circuitry 25 uses the answer acquisition function 254 to acquire the solutions output from the language generation model 211. Here, it is assumed that multiple solutions have been output in response to the question "How to put the bed in an upright position."

[0054] (Step S106) Next, the processing circuitry 25 executes a determination process using the determination function 255 to determine the feasibility of each solution output from the language generation model 211 on its own device. Figure 6 is a flowchart showing an example of the procedure of the determination process executed by the processing circuitry 25 in the determination function 255.

[0055] (Determination process) (Step S111) In the determination process, the processing circuitry 25 first determines, in step S111, for each solution output from the language generation model 211, whether the content of the solution is related to the operation of the device itself. At this time, the processing circuitry 25 performs natural language processing on each solution and extracts keywords from the solution. Then, the processing circuitry 25 calculates the similarity between the extracted keywords and the manual for the device itself. If the similarity between the keywords and the manual is high (step S111-Yes), the processing circuitry 25 determines that the solution is related to the operation of the device itself. On the other hand, if the similarity between the keywords and the manual is low (step S111-No), the processing circuitry 25 determines that the solution is not related to the operation of the device itself, and excludes the solution from the display candidates. Solutions excluded from the display candidates are not presented to the user, and feasibility is not evaluated.

[0056] (Step S112) Next, the processing circuitry 25 determines whether the solution is feasible on the device itself. At this time, the processing circuitry 25 performs a simulation of executing the solution on the device itself. If an error or a potential bug occurs in the simulation results, the processing circuitry 25 determines that the solution is not feasible on the device itself. For example, the processing circuitry 25 executes an interference control program pre-stored in the memory 21 along with the solution, and checks whether a predetermined error or an unintended bug occurs when the X-ray diagnostic device 1 is operated according to the solution. For example, if the device itself has more moving parts than a typical X-ray diagnostic device, an error occurs when the moving part to be operated cannot be identified using the operation method included in the solution, or when the moving part to be operated interferes with a moving part not installed in a typical model. If no error or bug occurs, the processing circuitry 25 determines that the solution is feasible on the device itself (step S112—Yes). On the other hand, if an error or bug occurs, the processing circuitry 25 determines that the solution is not feasible on the device itself (step S112—No) and excludes the solution from the display candidates. Solutions excluded from the display candidates are not presented to the user, and their feasibility is not evaluated.

[0057] (Step S113) Furthermore, if an error or bug occurs in the processing of step S112 (step S112-No), the processing circuitry 25 stores the details of the occurred error or bug as a log in the memory 21. The error or bug recorded as the log can be used for training the language generation model 211 or for maintenance of the X-ray diagnostic apparatus 1.

[0058] (Step S114) Next, the processing circuitry 25 determines whether the solution takes into account the current situation of the apparatus. At this time, the processing circuitry 25 determines whether the solution is applicable to the current situation of the apparatus based on situation information regarding the current situation of the apparatus. The situation information includes the current positions of each axis of the X-ray diagnostic apparatus 1, the operation status of the X-ray diagnostic apparatus 1, the operating status of the X-ray diagnostic apparatus 1, the current arrangement of people and objects in the examination room, and examination information for patients scheduled for examination. The current arrangement of people and objects in the examination room is acquired, for example, by analyzing camera images taken inside the examination room. The situation information is stored in advance in the memory 21, for example. The current arrangement of people and objects in the examination room may also be acquired by analyzing audio data acquired from a microphone installed in the examination room.

[0059] Furthermore, status information of the X-ray diagnostic apparatus 1 may be acquired using digital twin technology. In this case, for example, a digital twin may be created using the operation history of the apparatus itself. The operation history includes, for example, the operation history, manipulation history, and operating status of each axis. The operating status includes, for example, the usage time, number of days of usage, and frequency of use. The operation history of each axis is acquired using sensor values ​​from, for example, an encoder. By creating a digital twin using the operation history of the apparatus itself, the current status of the apparatus itself is reflected in the digital twin. When acquiring status information using digital twin technology, information regarding the internal state of the X-ray diagnostic apparatus 1 can be used, making it possible to acquire status information with a larger amount of information and higher accuracy than when acquiring status information using camera images.

[0060] For example, the processing circuitry 25 uses the simulation results executed in step S112 to determine whether there is a risk of the moving parts of the X-ray diagnostic apparatus 1 colliding with a person or object in the examination room if the apparatus is operated according to the solution, and if there is a risk of collision, determines that the solution is inapplicable to the current situation of the apparatus. Alternatively, the processing circuitry 25 may calculate the time or man-hours required to execute the operation of the solution on the apparatus based on the current axial position of the X-ray diagnostic apparatus 1, and if the time or man-hours exceed a predetermined value, determine that the solution is inapplicable to the current situation of the apparatus. Alternatively, the processing circuitry 25 may search for operations that would be a burden on the patient from among the operations included in the solution based on the patient's examination information, and if an operation that would be a burden on the patient is included, determine that the solution is inapplicable to the current situation of the apparatus.

[0061] If the solution is applicable to the current situation of the own device, the processing circuitry 25 determines that the solution takes into account the current situation of the own device (step S114-Yes). On the other hand, if the solution is not applicable to the current situation of the own device, the processing circuitry 25 determines that the solution does not take into account the current situation of the own device (step S114-No) and excludes the solution from the display candidates. Solutions excluded from the display candidates are not presented to the user, and their feasibility is not evaluated.

[0062] (Step S115) Next, the processing circuitry 25 determines whether a state (hereinafter referred to as a post-operation state) resulting from the implementation of the solution proposed by the language generation model 211 satisfies the conditions of the desired state. In this case, the processing circuitry 25 predicts the post-operation state and compares the post-operation state with the desired state predicted in step S102 to determine the feasibility of the solution. The post-operation state may be predicted using the results of the simulation executed in step S112, or a new simulation for the desired state may be executed. For example, if the desired state is "a state in which the bed rotation angle is greater than or equal to 80° and less than or equal to 90°" and the post-operation state is "a state in which the bed rotation angle is approximately 89°," the processing circuitry 25 determines that the post-operation state is a solution that satisfies the conditions of the desired state (step S115—Yes). On the other hand, if the post-operation state does not satisfy the conditions of the desired state (step S115—No), the processing circuitry 25 excludes the solution from the display candidates. Solutions excluded from the display candidates are not presented to the user, and their feasibility is not evaluated.

[0063] (Step S116) Next, the processing circuitry 25 evaluates the feasibility of the solution based on the determinations made in steps S112 to S115. For example, the determinations made in each step S112 to S115 are quantified based on a predetermined rule, and the numerical values ​​are added together to calculate the evaluation value of the feasibility. For example, if the examination information indicates that the patient will be undergoing CT imaging while sitting in a wheelchair, assistance will be required to perform the imaging in an upright position, so the evaluation value of the feasibility should be lowered.

[0064] (Step S117) Next, the processing circuit 25 determines whether or not the determination of feasibility has been completed for all of the solutions proposed by the language generation model 211. If there are any solutions for which the determination of feasibility has not been completed (step S117-No), the process returns to step S111.

[0065] The processing circuit 25 repeatedly executes the processes of steps S111 to S116 for each solution and calculates the evaluation value for all solutions that have not been excluded from the display candidates. If the above processes have been executed for all solutions (step S117-Yes), the processing circuit 25 ends the determination process in step S106.

[0066] (Step S107) When the determination process is completed, the processing circuitry 25 causes the display control function 256 to display on the display 23 the solutions remaining as display candidates as answers to the question. At this time, the processing circuitry 25 causes the solutions to be displayed in the answer display section 232 of the operation assistance screen. FIGS. 7 and 8 are diagrams showing examples of answers displayed in the answer display section 232. FIG. 8 shows an example of a display when the display in the answer display section 232 of FIG. 7 is scrolled down. In FIG. 7, the answer display section 232 displays the type of checks made in determining the feasibility in step S107. For example, the answer display section 232 displays the results of the feasibility determinations in steps S111 to S115 and information used in the determinations. For example, the type, drawing number, page number, etc. of the manual referred to in the simulation are displayed in the answer display section 232. Alternatively, if there is a risk of contact with a surrounding installation, a photo of the installation may be displayed in the answer display section 232.

[0067] 8, the solutions remaining as display candidates are displayed in descending order of evaluation value in the answer display section 232. At this time, the answer display section 232 does not display solutions that have been excluded from the display candidates in the processes of steps S111 to S115.

[0068] The effects of the X-ray diagnostic apparatus 1 according to this embodiment will be described below.

[0069] The X-ray diagnostic apparatus 1 equipped with the medical information processing apparatus of this embodiment acquires a question regarding the operation of the apparatus as a user input, inputs the acquired question into a language generation model 211 that has been trained to output an answer to the question when it receives input of a question regarding the operation of the apparatus, acquires the output of the language generation model 211, determines the feasibility of the output acquired from the language generation model on the apparatus itself, and displays information corresponding to the output on the display 23 depending on the feasibility.

[0070] With the above configuration, the X-ray diagnostic apparatus 1 of this embodiment has the function of extracting and answering a question about a user's operation using a language generation AI such as chatGPT (registered trademark). Furthermore, before answering the solution to the user, the X-ray diagnostic apparatus 1 of this embodiment can verify the feasibility of the solution extracted by the language generation AI on its own apparatus, confirm the validity of the solution, and propose a solution whose validity has been confirmed. This allows the X-ray diagnostic apparatus 1 to provide accurate support for the operation desired by the user, allowing the user to perform the operation efficiently. Furthermore, by using the language generation AI, solutions including multiple operations can be presented to the user in an appropriate order.

[0071] For example, for each solution proposed by the language generation AI, it can be confirmed whether the solution is related to the operation of the device, whether it is feasible on the device, whether it takes into account the current status of the device, and whether it achieves the user's desired final goal, and the feasibility can be evaluated based on the confirmation results. For example, the feasibility of each item can be confirmed using the results of a simulation of the operation when the solution is executed on the device. Note that in this embodiment, determinations are made on four determination items in the processing of steps S111 to S115, but it is sufficient to determine the feasibility of one or more of the above items, and determination of some items may be omitted.

[0072] In addition, in this embodiment, solutions that have been excluded from the display candidates are not displayed as answers, so the user can confirm only the solutions that are feasible on the user's device by checking only the solutions that remain in the display candidates. In addition, in this embodiment, solutions are displayed in descending order of feasibility evaluation value, so the user can prioritize checking the appropriate solution by checking the displayed solutions in order from top to bottom.

[0073] (Variation) In this embodiment, the feasibility evaluation value is calculated for only the solutions remaining in the display candidates, but it is also possible to calculate the feasibility evaluation value for all solutions and display only solutions with evaluation values ​​equal to or greater than a predetermined value.Alternatively, it is also possible to calculate the feasibility evaluation value for all solutions and display all solutions in descending order of evaluation value.

[0074] 9, in addition to the solutions, an evaluation value of the feasibility of each solution may be displayed. In this case, the display control function 256 of the processing circuit 25 causes the display 23 to display the feasibility of each solution.

[0075] Additionally, the situation information used to determine the feasibility and information regarding risks detected based on the situation information may be displayed together with the solution. For example, when displaying situation information for each solution, the display control function 256 of the processing circuitry 25 may display the time and man-hours required to execute the solution on the device itself together with the solution. Alternatively, operations that are burdensome to the patient and are extracted from among the operations included in the solution may be displayed together with the solution to prompt the user's attention and response.

[0076] In the above embodiment, the medical information processing device 30 is described as being installed in the X-ray diagnostic device 1, but the medical information processing device can be applied to a device having a function of answering questions about how to operate the device. The medical information processing device may be installed in a medical device other than a medical diagnostic device, or in a device other than a medical device.

[0077] For example, a medical information processing device may be installed on a general PC terminal or smartphone terminal that has the function of answering questions about operation using a language generation AI. In this case, multiple solutions to questions about how to operate the terminal are obtained from the language generation AI, the feasibility of each solution is determined, and the solution is presented to the user based on its feasibility. The feasibility determination can be made using, for example, an online manual or the support page of the device's sales company. In this case, in addition to the solution, links and images from the online information referenced when determining the feasibility may be displayed. Furthermore, digital twin technology may be used to simulate the operation of a solution when executed on the device, and the feasibility may be determined. In this case, a simulation video using digital twin technology may be displayed together with the solution. For example, the simulation video may be generated to show cursor movement and screen changes when the operations included in the solution are executed.

[0078] According to at least one of the embodiments described above, it is possible to efficiently provide support to a user regarding the operation of a medical device.

[0079] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0080] 1...X-ray diagnostic equipment 3...X-ray tube 5...Irradiation range limiter 7...X-ray detector 9...Movement support mechanism 11...Rotation support mechanism 13...Drive unit 14...Tabletop 16...Output section 16a...Speaker 16b...Display 20...Console device 21...Memory 211...Language generation model 22...Input interface 23...Display 231...Question input section 232…Answer display section 24...Communication interface 25...Processing circuit 251...Control function 252...Image processing function 253...Input acquisition function 254…Answer acquisition function 255...Determination function 256...Display control function 30...Medical information processing device

Claims

1. an acquisition unit that acquires user input related to the operation of the medical device; an answer acquisition unit that inputs the acquired user input into a trained model that has been trained to output an answer to the user input when receiving the user input related to the operation of the medical device, and acquires an output of the trained model; a determination unit that determines the feasibility of the acquired output on the device itself; a display control unit that displays information corresponding to the output on a display in accordance with the feasibility; A medical information processing device comprising:

2. the determination unit executes a simulation of the case where the output is applied to the device itself, and determines the feasibility of the output based on the simulation result. The medical information processing device according to claim 1 .

3. The determination unit determines that the feasibility of the output is high when the simulation result satisfies a predetermined condition. The medical information processing device according to claim 2 .

4. The determination unit determines that the feasibility of the output is low when the simulation result includes an error or a bug. The medical information processing device according to claim 3 .

5. If the simulation result includes an error or a bug, the determination unit stores the error or the bug as a log. The medical information processing device according to claim 4 .

6. the determination unit determines the feasibility of output based on situation information including location information of the device itself, an operation situation, an operating situation, or location information of a surrounding object; The medical information processing device according to claim 1 .

7. The medical information processing apparatus according to claim 6 , wherein the determining unit executes a simulation of a case where the output is applied to the apparatus itself based on the status information, and determines the feasibility of the output based on a result of the simulation.

8. The medical information processing apparatus according to claim 6 , wherein the display control unit causes the display to display the status information together with the output.

9. the determination unit determines whether the output is information relating to an operation of the device itself, and determines the feasibility of the output based on a determination result. The medical information processing device according to claim 1 .

10. the determination unit estimates a target state desired by the user based on the user input, estimates a post-operation state when the output is implemented, and determines the feasibility of implementing the output by comparing the post-operation state with the target state; The medical information processing device according to claim 1 .

11. the trained model outputs an answer including multiple solutions for inputs including one user input; the display control unit displays only solutions with high feasibility among the plurality of solutions, The medical information processing device according to claim 1 .

12. the trained model outputs an answer including multiple solutions for inputs including one user input; the display control unit displays the plurality of solutions in descending order of feasibility; The medical information processing device according to claim 1 .

13. the trained model outputs an answer including multiple solutions for inputs including one user input; the display control unit displays the plurality of solutions and feasibility of each solution. The medical information processing device according to claim 1 .

14. The medical information processing device according to claim 1 ; A storage unit that stores the trained model; an image processing unit that generates a medical image of the subject; an input unit to which the user input is input; the display that displays information according to the output; A medical diagnostic device comprising:

15. the medical diagnostic device is an X-ray diagnostic device, an X-ray CT device, an MRI device, or an ultrasound diagnostic device; 15. The medical diagnostic device of claim 14.

16. the X-ray diagnostic apparatus is an X-ray TV bed apparatus; 16. The medical diagnostic device of claim 15.

17. The determination unit determines the feasibility of the output using a digital twin. The medical information processing device according to claim 1 .

18. The display control unit displays the results of the digital twin on the display. The medical information processing device according to claim 17.

19. A medical information processing method using the medical information processing device according to claim 1, the acquiring unit acquiring the user input; The answer acquisition unit inputs the acquired user input into the trained model and acquires an output of the trained model; the determination unit determines the feasibility of the acquired output; the display control unit causes the display to display information corresponding to an output with a high feasibility; A medical information processing method comprising:

Images