Medical information processing apparatus, medical information processing method, and program

Abstract

To provide a medical information processing apparatus, a medical information processing method and a program that enable a user to comfortably observe and operate images on a terminal device.SOLUTION: A medical information processing apparatus according to an embodiment includes an acquisition unit and a transmission control unit. The acquisition unit acquires an operation history of a user with respect to medical information which is generated by a medical imaging diagnostic apparatus and is transmitted to and displayed on a terminal device located remotely from the medical imaging diagnostic apparatus. The transmission control unit controls a transmission mode of the medical information transmitted from the medical imaging diagnostic apparatus to the terminal device based on the operation history.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The embodiments disclosed in the present specification and drawings relate to a medical information processing device, a medical information processing method, and a program. [Background technology]

[0002] There is known a technology for transmitting medical images generated by an external medical image diagnostic device (also called a modality) to a remote terminal device and displaying them. Examples of the medical image diagnostic device include an ultrasound diagnostic device, an X-ray CT (Computed Tomography) device, and an MRI (Magnetic Resonance Imaging) device. The medical image diagnostic device may also be a biological monitor that measures biological information such as an electrocardiogram, respiration, body temperature, and blood pressure.

[0003] Generally, when medical images or biological information obtained by a medical image diagnostic device are remotely transmitted to a terminal device, a large delay may occur in the image displayed on the terminal device due to various factors such as transmission control, communication bandwidth, communication path, and signal processing. In such a case, when remotely operating the terminal device, the large delay may cause a delay in the displayed image in response to an operation from the terminal device, resulting in operational errors or a deterioration in operability. As a result, a user may not be able to comfortably observe or operate the image on the terminal device. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-136112 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the embodiments disclosed in this specification and the drawings is to enable a user to comfortably observe and operate images on a terminal device. However, the problem to be solved by the embodiments disclosed in this specification and the drawings is not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0006] The medical information processing apparatus of the present embodiment includes an acquisition unit and a transmission control unit. The acquisition unit acquires a user's operation history for medical information generated by a medical image diagnostic apparatus, which is transmitted to and displayed on a terminal device remote from the medical image diagnostic apparatus. The transmission control unit controls a transmission mode of the medical information transmitted from the medical image diagnostic apparatus to the terminal device based on the operation history. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram showing an example of the configuration of a medical information processing system 1 according to a first embodiment. [Figure 2] 1 is a diagram showing an example of the arrangement of an ultrasound diagnostic apparatus 100 according to a first embodiment. [Figure 3] FIG. 2 is a diagram showing an example of the configuration of a server 200 according to the first embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of the configuration of a terminal device 300 according to the first embodiment. [Figure 5] FIG. 2 is a sequence diagram showing an example of a series of processing flows in the medical information processing system 1 according to the first embodiment. [Figure 6] 4 is a flowchart showing an example of the flow of a series of processes of the server 200 according to the first embodiment. [Figure 7] FIG. 4 is a diagram for explaining a method for detecting a menu area R. [Figure 8] 10A and 10B are diagrams for explaining an example of a user operation when it is predicted that the user intends to operate a menu. [Figure 9]10A and 10B are diagrams for explaining another example of a user operation when it is predicted that the user intends to operate the menu. [Figure 10] 10A and 10B are diagrams for explaining an example of a user operation when it is predicted that the user has no intention of operating a menu. [Figure 11] FIG. 10 is a diagram for explaining another example of a user operation when it is predicted that the user has no intention of operating the menu. [Figure 12] 10A and 10B are diagrams for explaining another example of a user operation when it is predicted that the user intends to operate the menu. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, a medical information processing apparatus, a medical information processing method, and a program according to an embodiment will be described with reference to the drawings.

[0009] (First embodiment) [Configuration of medical information processing system] 1 is a diagram illustrating an example of the configuration of a medical information processing system 1 according to the first embodiment. The medical information processing system 1 includes, for example, a plurality of medical image diagnostic devices (modalities) 100, a server 200, and a terminal device 300. These devices are communicably connected via a communication network NW.

[0010] The communication network NW may refer to any information and communication network that uses telecommunications technology. For example, the communication network NW may include wireless / wired LANs such as hospital backbone LANs (Local Area Networks), the Internet, telephone communication networks, optical fiber communication networks, cable communication networks, satellite communication networks, etc.

[0011] The medical image diagnostic device 100 includes modalities such as an ultrasound diagnostic device, an X-ray CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, an X-ray device, etc. The medical image diagnostic device 100 may also include a biological monitor that measures biological information such as an electrocardiogram, respiration, body temperature, and blood pressure.

[0012] The medical image diagnostic apparatus 100 is installed at a location away from the terminal device 300. For example, if the medical image diagnostic apparatus 100 is installed in an operating room of a hospital, the terminal device 300 may be installed in an examination room of the same hospital or in another medical institution. In the following, as an example, the medical image diagnostic apparatus 100 will be described as an ultrasound diagnostic apparatus.

[0013] The server 200 receives medical information from the ultrasound diagnostic device (medical image diagnostic device) 100 and receives user operation history from the terminal device 300 via the communication network NW. The medical information may include medical images such as ultrasound images and various information associated therewith (such as menus described below). The medical images may be still images or moving images (i.e., video). In the following description, when the medical images are moving images (video), they will be specifically referred to as medical videos.

[0014] When the server 200 receives the medical information and the operation history, it determines a transmission mode for the medical information and transfers the medical information with image quality and / or delay according to the transmission mode to the terminal device 300. The server 200 is an example of a "medical information processing device."

[0015] The server 200 may be a single device, or may be a system in which multiple devices connected via a communication network NW operate in cooperation with each other. That is, the server 200 may be realized by multiple computers (processors) included in a distributed computing system or a cloud computing system. Furthermore, the server 200 does not necessarily have to be a separate device from the ultrasound diagnostic apparatus (medical image diagnostic apparatus) 100 or the terminal device 300, but may be a device integrated with either the ultrasound diagnostic apparatus (medical image diagnostic apparatus) 100 or the terminal device 300. In this case, the server 200 integrated with the ultrasound diagnostic apparatus (medical image diagnostic apparatus) 100 or the server 200 integrated with the terminal device 300 is another example of a "medical information processing device."

[0016] The terminal device 300 is a computer that can be operated by a user such as a doctor, technician, nurse, etc. The terminal device 300 displays medical information transmitted from the server 200 and collects the operation history of the user.

[0017] [Configuration of ultrasound diagnostic equipment] 2 is a diagram showing an example of the configuration of an ultrasound diagnostic apparatus 100 according to the first embodiment. The ultrasound diagnostic apparatus 100 is an apparatus that generates a medical image of the subject P (hereinafter referred to as an ultrasound image) by scanning the subject P with ultrasound, and diagnoses the subject P based on the ultrasound image. The subject P is typically a human, but is not limited to this, and may be another animal such as a dog or a cat, or may be a plant.

[0018] The ultrasound diagnostic device 100 includes, for example, a communication interface 110, an input interface 120, an output interface 130, an ultrasound probe 140, a processing circuit 150, and a memory 160.

[0019] The communication interface 110 includes, for example, a network interface card (NIC) and an antenna for wireless communication. The communication interface 110 communicates with an external device via a communication network NW. The external device is the above-mentioned server 200, terminal device 300, or the like.

[0020] The input interface 120 accepts various input operations from the user, converts the accepted input operations into electrical signals, and outputs the electrical signals to the processing circuit 150. The input interface 120 includes physical operation components such as a mouse, keyboard, trackball, switches, buttons, joystick, and touch panel. When the input interface 120 is a touch panel, the input interface 120 may also have the display function of the display 132, which will be described later.

[0021] Furthermore, in this specification, the input interface 120 is not limited to an interface having physical operation parts such as a mouse, keyboard, etc. For example, an example of the input interface 120 also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to a control circuit.

[0022] The output interface 130 includes, for example, a display 132 and a speaker 134. The display 132 displays various types of information. For example, the display 132 displays information output by the processing circuit 150 as an image, and displays a GUI (Graphical User Interface) for receiving various input operations from the user. For example, the display 132 is an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display, or the like. The speaker 134 outputs the information output by the processing circuit 150 as sound.

[0023] The ultrasonic probe 140 is operated by a user such as a doctor, technician, or nurse, and pressed against a part of the subject P (a region to be examined or diagnosed). The ultrasonic probe 140 transmits (irradiates) ultrasonic waves to the subject P, for example, to acquire an image of the inside of the subject P. The ultrasonic probe 140 receives echoes (reflected waves) of the transmitted ultrasonic waves. The ultrasonic probe 140 then generates signal data of the received echoes (hereinafter referred to as echo data) and outputs the echo data. For example, the ultrasonic probe 140 may be a two-dimensional array probe in which a plurality of vibrators (transducers) that transmit and receive ultrasonic waves are arranged in a two-dimensional array. In this case, the echo data may be three-dimensional volume data in which the signal intensity of the echo is associated with each unit area obtained by dividing a three-dimensional space having width, height, and depth.

[0024] The processing circuitry 150 generates an ultrasound image from the echo data generated by the ultrasound probe 140. Then, the processing circuitry 150 transmits (sends) to the server 200 via the communication interface 110 the medical information including the ultrasound image and accompanying information such as a menu for allowing the user to specify conditions for generating and displaying the ultrasound image.

[0025] The processing circuitry 150 realizes these functions by, for example, a hardware processor (computer) executing a program stored in a memory 160 (storage circuitry).

[0026] The hardware processor in the processing circuit 150 refers to a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD) or a complex programmable logic device (CPLD), or a field programmable gate array (FPGA)). Instead of storing the program in the memory 160, the program may be directly embedded in the circuit of the hardware processor. In this case, the hardware processor realizes its function by reading and executing the program embedded in the circuit. The program may be stored in the memory 160 in advance, or may be stored in a non-transitory storage medium such as a DVD or CD-ROM, and installed in the memory 160 from the non-transitory storage medium when the non-transitory storage medium is inserted into a drive device (not shown) of the ultrasound diagnostic apparatus 100. The hardware processor is not limited to being configured as a single circuit, but may be configured as a single hardware processor by combining multiple independent circuits to realize each function, or multiple components may be integrated into a single hardware processor to realize each function.

[0027] The memory 160 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory), a flash memory, a hard disk, or an optical disk. These non-transitory storage media may also be realized by other storage devices connected via a communication network NW, such as a NAS (Network Attached Storage) or an external storage server device. The memory 160 may also include other non-transitory storage media such as a ROM (Read Only Memory) or a register.

[0028] [Server Configuration] 3 is a diagram illustrating an example of the configuration of the server 200 according to the first embodiment. The server 200 includes, for example, a communication interface 210, a memory 220, and a processing circuit 230.

[0029] The communication interface 210 communicates with external devices via the communication network NW. The communication interface 210 includes, for example, a NIC, an antenna for wireless communication, etc. The external devices are the ultrasound diagnostic device 100 and the terminal device 300 described above.

[0030] The memory 220 is realized by, for example, a semiconductor memory element such as RAM or flash memory, a hard disk, or an optical disk. These non-transitory storage media may also be realized by other storage devices connected via a communication network NW, such as a NAS or an external storage server device. The memory 220 may also include other non-transitory storage media such as ROM or registers.

[0031] The processing circuit 230 includes, for example, an acquisition function 231, a transmission control function 232, an operation detection function 233, and an output control function 234. The acquisition function 231 and / or the operation detection function 233 are an example of an "acquisition unit." The transmission control function 232 is an example of a "transmission control unit."

[0032] The processing circuitry 230 realizes these functions by, for example, a hardware processor (computer) executing a program stored in the memory 220 (storage circuitry).

[0033] The hardware processor in the processing circuit 230 refers to a circuit such as a CPU, a GPU, an application-specific integrated circuit, or a programmable logic device (e.g., a simple programmable logic device, a composite programmable logic device, or a field programmable gate array). Instead of storing a program in the memory 220, the program may be directly embedded in the circuit of the hardware processor. In this case, the hardware processor realizes its functions by reading and executing the program embedded in the circuit. The program may be pre-stored in the memory 220, or may be stored on a non-transitory storage medium such as a DVD or CD-ROM. The program may be installed in the memory 220 from the non-transitory storage medium by inserting the non-transitory storage medium into a drive (not shown) of the server 200. The hardware processor is not limited to being configured as a single circuit, but may be configured as a single hardware processor by combining multiple independent circuits to realize each function. Furthermore, multiple components may be integrated into a single hardware processor to realize each function.

[0034] The acquisition function 231 acquires medical information from the ultrasound diagnostic apparatus 100 via the communication interface 210 .

[0035] Furthermore, the acquisition function 231 acquires, from the terminal device 300 via the communication interface 210, the user's operation history for the terminal device 300 on which the medical video (that is, the moving image of the ultrasound image) included in the medical information is displayed.

[0036] The operation history may include the movement trajectory of a pointer (also called a cursor) of a pointing device such as a mouse or a touch panel, a position selected by an operator on a GUI (for example, a position clicked with a mouse), etc. Furthermore, if the pointer is operated by the user's voice, line of sight, or gesture instead of a pointing device, the operation history may also include the movement trajectory of the pointer that moves in accordance with the user's voice, line of sight, or gesture.

[0037] The transmission control function 232 controls the mode (hereinafter referred to as the transmission mode) when transferring (transmitting) medical information acquired from the ultrasound diagnostic device 100 to the terminal device 300. The transmission modes include at least a low-delay mode and a high-image-quality mode.

[0038] The low-delay mode is a mode for prioritizing transmission speed over image quality, and is a mode in which medical images are displayed on the terminal device 300 with low delay (a mode in which the delay speed or delay time is less than a threshold).

[0039] The high-quality mode is a mode in which image quality takes priority over transmission speed, and in which the image quality of the medical video displayed on the terminal device 300 is higher than in the low-delay mode.

[0040] The operation detection function 233 detects a specific operation history for switching the transmission mode from the user operation history acquired from the terminal device 300 by the acquisition function 231. The specific operation history includes an operation for switching the transmission mode from the high-image-quality mode to the low-latency mode and an operation for switching the transmission mode from the low-latency mode to the high-image-quality mode. Details of these operations will be described later.

[0041] The output control function 234 transmits (transmits) the medical information acquired from the ultrasound diagnostic device 100 to the terminal device 300 via the communication interface 210 at an image quality and transmission speed according to the transmission mode.

[0042] [Terminal device configuration] 4 is a diagram illustrating an example of the configuration of the terminal device 300 according to the first embodiment. The terminal device 300 includes, for example, a communication interface 310, an input interface 320, an output interface 330, a processing circuit 340, and a memory 350.

[0043] The communication interface 310 includes, for example, a NIC, an antenna for wireless communication, etc. The communication interface 310 communicates with an external device via a communication network NW. The external device is the ultrasound diagnostic device 100, the server 200, or the like.

[0044] The input interface 320 accepts various input operations from the user, converts the accepted input operations into electrical signals, and outputs the electrical signals to the processing circuit 340. The input interface 320 is, for example, a mouse 320a, but is not limited to this, and may include physical operation components such as a keyboard, a trackball, a switch, a button, a joystick, and a touch panel. If the input interface 320 is a touch panel, the input interface 320 may also have the display function of the display 332 described below.

[0045] The input interface 320 may be a user interface that includes, for example, a microphone and accepts voice input from the user, or may be a user interface that includes, for example, a camera or a motion capture sensor and accepts the user's gaze or gestures as input.

[0046] Furthermore, in this specification, the input interface 320 is not limited to an interface having physical operation parts such as a mouse, keyboard, etc. For example, an example of the input interface 320 also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to a control circuit.

[0047] The output interface 330 includes, for example, a display 332 and a speaker 334. The display 332 displays various types of information. For example, the display 332 displays information output by the processing circuit 340 as an image, or displays a GUI for receiving various input operations from the user. For example, the display 332 is an LCD, an organic EL display, or the like. The speaker 334 outputs the information output by the processing circuit 340 as sound.

[0048] The processing circuitry 340 receives medical information from the server 200 via the communication interface 310 and outputs the medical information via the output interface 330. The processing circuitry 340 also collects operation histories of users (doctors, engineers, nurses, etc.) that have input to the input interface 320 and transmits them to the server 200 via the communication interface 310.

[0049] The processing circuitry 340 realizes these functions by, for example, a hardware processor (computer) executing a program stored in a memory 350 (storage circuitry).

[0050] The hardware processor in the processing circuit 340 refers to a circuit such as a CPU, a GPU, an application-specific integrated circuit, or a programmable logic device (e.g., a simple programmable logic device, a composite programmable logic device, or a field programmable gate array). Instead of storing a program in the memory 350, the program may be directly embedded in the circuit of the hardware processor. In this case, the hardware processor realizes its functions by reading and executing the program embedded in the circuit. The program may be pre-stored in the memory 350, or may be stored on a non-transitory storage medium such as a DVD or CD-ROM. The program may be installed in the memory 350 from the non-transitory storage medium by inserting the non-transitory storage medium into a drive (not shown) of the terminal device 300. The hardware processor is not limited to being configured as a single circuit, but may be configured as a single hardware processor by combining multiple independent circuits to realize each function. Furthermore, multiple components may be integrated into a single hardware processor to realize each function.

[0051] The memory 350 is realized by, for example, a semiconductor memory element such as RAM or flash memory, a hard disk, or an optical disk. These non-transitory storage media may also be realized by other storage devices connected via a communication network NW, such as a NAS or an external storage server device. The memory 350 may also include other non-transitory storage media such as ROM or registers.

[0052] [Sequence of medical information processing system] First, a sequence diagram will be used to explain the overall processing flow of the medical information processing system 1. Fig. 5 is a sequence diagram showing an example of the flow of a series of processes of the medical information processing system 1 according to the first embodiment.

[0053] First, an ultrasound diagnostic device 100, which is a type of medical image diagnostic device, scans a subject P and generates an ultrasound image (step S100).

[0054] Next, the ultrasound diagnostic device 100 transmits medical information including the ultrasound image as a medical video to the server 200 (step S102). In the subsequent processing, as the scan of the subject P continues and a new ultrasound image is generated, the new ultrasound image is transmitted from the ultrasound diagnostic device 100 to the server 200 as one frame of the medical video.

[0055] Next, when the server 200 receives the medical information from the ultrasound diagnostic device 100, it initializes the transmission mode (step S104) and transmits the medical information to the terminal device 300 in accordance with the initialized transmission mode (hereinafter, referred to as high-image-quality mode as an example) (step S106).

[0056] Next, when the terminal device 300 receives the medical information transmitted in the high-image-quality mode from the server 200, it displays it on the display 332 (step S108).

[0057] Next, the terminal device 300 transmits to the server 200 a history of user operations input to the input interface 320 while the medical information transmitted in the high-quality image mode is being displayed on the display 332 (step S110).

[0058] Next, when the server 200 receives the user's operation history from the terminal device 300, it switches the transmission mode of the medical information to be transmitted to the terminal device 300 next based on the user's operation history (step S112). Here, it is assumed that the transmission mode is switched from the initialized high-image-quality mode to the low-latency mode.

[0059] Next, the server 200 transmits the medical information to the terminal device 300 in the low latency mode (step S114).

[0060] Next, when the terminal device 300 receives the medical information transmitted in the low-delay mode from the server 200, it displays it on the display 332 (step S116).

[0061] Next, the terminal device 300 transmits to the server 200 a history of user operations input to the input interface 320 while the medical information transmitted in the low-latency mode is being displayed on the display 332 (step S118).

[0062] Next, when the server 200 receives the user's operation history from the terminal device 300, it switches the transmission mode of the medical information to be transmitted to the terminal device 300 next based on the user's operation history (step S120). Here, it is assumed that the mode is switched again from the low latency mode to the high image quality mode.

[0063] Next, the server 200 transmits the medical information to the terminal device 300 in the high-image-quality mode (step S122).

[0064] In this way, during the series of processes of transmitting medical information from the server 200 to the terminal device 300, the medical information processing system 1 switches the transmission mode from high-quality mode to low-latency mode, or from low-latency mode to high-quality mode, depending on the user's operation history.

[0065] [Server processing flow] Next, a flowchart will be used to explain the flow of processing performed by the server 200 alone. Fig. 6 is a flowchart showing an example of the flow of a series of processing performed by the server 200 according to the first embodiment.

[0066] First, the transmission control function 232 of the processing circuit 230 initializes the transmission mode (step S200). As described above, the initialized transmission mode is assumed to be the high-quality mode, for example.

[0067] Next, the operation detection function 233 determines whether or not medical information including medical images has been received from the ultrasound diagnostic apparatus 100 via the communication interface 210 (step S202).

[0068] When medical information including medical images is received from the ultrasound diagnostic device 100 via the communication interface 210, the operation detection function 233 detects an area on the medical images (medical images) where a menu exists (hereinafter referred to as menu area R) (step S204). As described above, a menu is an item that allows the user to specify conditions for generating and displaying a medical image such as an ultrasound image.

[0069] FIG. 7 is a diagram for explaining a method for detecting a menu area R. In the illustrated example, medical image IMGs from a plurality of medical image diagnostic devices 100 are displayed in thumbnail format as medical information to be transmitted to the terminal device 300. For example, the medical image IMGs may be ultrasound images from an ultrasound diagnostic device, CT images from an X-ray CT device, MR images from an MRI device, or X-ray images from an X-ray device. The operation detection function 233 detects a menu area R for each of these plurality of medical image IMGs. In the illustrated example, the menu area R is detected for IMG4 and IMG6. Note that the medical information transmitted to the terminal device 300 may be only a single medical image IMG.

[0070] Returning to the description of the flowchart, the output control function 234 then transmits the medical information including the medical image to the terminal device 300 via the communication interface 210 in accordance with the high-quality mode, which is the initialized transmission mode (step S206).

[0071] For example, the output control function 234 may increase the frame rate and transmit the medical image, which is volume data, with high image quality. The output control function 234 may also use other existing technologies to improve the image quality of the medical image.

[0072] Next, the operation detection function 233 determines whether or not the user has performed an end operation on the terminal device 300 that displays the medical video transmitted in the high-quality mode (step S208).

[0073] For example, suppose that the acquisition function 231 acquires a user's operation history for the terminal device 300 that displays a medical image transmitted in high-definition mode. In response to this, the operation detection function 233 determines whether or not the user's operation history includes an end operation.

[0074] An end operation may be, for example, clicking the close button or minimize button located in the upper right corner of the medical image in which the menu area R is detected, clicking on another medical image, or no user operation for a certain period of time.

[0075] If it is determined that the user has performed an end operation (i.e., the user has no intention of continuing the operation), the output control function 234 stops transmitting medical information, including medical images, to the terminal device 300 (step S210).

[0076] On the other hand, if it is determined that the user has not performed the termination operation (i.e., the user intends to continue the operation), the operation detection function 233 further predicts (determines) whether the user intends to operate the menu on the medical image transmitted in high-definition mode (step S212).

[0077] For example, when an operation of moving the pointer relatively closer to the menu area R or an operation of moving the pointer within the menu area R is performed, the operation detection function 233 may predict that the user intends to operate the menu.

[0078] If it is predicted that the user intends to operate the menu, the transmission control function 232 switches the transmission mode from the high-quality mode to the low-delay mode (step S214).

[0079] In response to this, the output control function 234 may lower the frame rate and transmit the medical video, which is volume data, with low latency. The output control function 234 may also use other existing technologies to reduce the latency of the medical video.

[0080] FIG. 8 is a diagram illustrating an example of a user's operation when it is predicted that the user intends to operate the menu. In the diagram, R represents the menu area, as described above. Furthermore, P(t1) represents the pointer at time t1, and P(t2) represents the pointer at time t2, which is later than time t1. As shown in the diagram, as time progresses from t1 to t2, the pointer P approaches the menu area R. The movement trajectory of the pointer from P(t1) to P(t2) predicts that the user intends to operate the menu. The transmission mode is then switched to a low-latency mode, and the medical video displayed on the terminal device 300 exhibits low latency. As a result, the pointer movement on the display 330a can be accurately tracked, precisely as the user actually moves the input interface 320 (e.g., the mouse 320a) at hand. In other words, the user can comfortably operate the menu while observing the medical video without experiencing any delay.

[0081] FIG. 9 is a diagram illustrating another example of a user's operation when it is predicted that the user intends to operate the menu. In the illustrated example, the pointer moves from P(t1) to P(t2) within the menu area R. This pointer movement trajectory also predicts that the user intends to operate the menu. When the user intends to continue operating the menu in this way, the delay of the medical video displayed on the terminal device 300 is reduced. As a result, as with the above, the user can continue to operate the menu comfortably while observing the medical video without feeling any delay.

[0082] Returning to the explanation of the flowchart, on the other hand, if it is predicted that the user has no intention of operating the menu, the process of S214 is omitted and the high-quality image mode is maintained.

[0083] Next, the operation detection function 233 predicts (determines) whether or not the user intends to operate the menu on the medical image (step S216).

[0084] For example, the operation detection function 233 may predict that the user has no intention of operating the menu if an operation is performed in which the pointer moves relatively away from the menu area R or the pointer stops within the menu area R without moving.

[0085] If it is predicted that the user has no intention of operating the menu, the transmission control function 232 switches the transmission mode to the high-quality image mode if the transmission mode is the low-latency mode (step S218), and maintains the high-quality image mode if the transmission mode is the high-quality image mode.

[0086] In response to this, the output control function 234 may increase the frame rate and transmit the medical image, which is volume data, with high image quality. The output control function 234 may also use other existing technologies to improve the image quality of the medical image.

[0087] 10 is a diagram illustrating an example of a user's operation when it is predicted that the user has no intention of operating the menu. In the illustrated example, the pointer is present in the same position within the menu area R at both time t1 and time t2 (P(t1)=P(t2)). In other words, the pointer remains stationary within the menu area R without moving. This pointer movement trajectory predicts that the user has no intention of operating the menu. While the user is not operating in this way, the image quality of the medical video displayed on the terminal device 300 is improved. This makes it possible to provide the user with high-quality medical video.

[0088] FIG. 11 is a diagram illustrating another example of a user's operation when it is predicted that the user has no intention of operating the menu. In the illustrated example, as time progresses from t1 to t2, the pointer P moves away from the menu area R. This movement trajectory of the pointer from P(t1) to P(t2) is considered to be a temporary movement of the pointer by the user, rather than a menu operation, and therefore it is predicted that the user has no intention of operating the menu. In response to this, the transmission mode is switched to high-quality mode, and the medical video displayed on the terminal device 300 becomes high-quality. As a result, high-quality medical video can be provided to the user.

[0089] In this way, by repeatedly reducing the latency or increasing the image quality of the medical images transmitted to the terminal device 300 until the user performs the termination operation, the user can comfortably observe and operate the medical images on the terminal device 300.

[0090] According to the first embodiment described above, the processing circuitry 230 of the server 200 acquires a user's operation history for the terminal device 300 on which medical information including medical images is displayed. The processing circuitry 230 controls the transmission mode of the medical information transmitted to the terminal device 300 based on the user's operation history. By controlling the transmission mode in this manner, it is possible to reduce the delay and increase the image quality of the medical images transmitted to the terminal device 300. As a result, the user can comfortably observe and operate the medical images on the terminal device 300.

[0091] (Second embodiment) The second embodiment will be described below. In the above-described first embodiment, the user's operation when it is predicted that the user intends to operate the menu is an operation in which the pointer moves relatively closer to the menu area R or an operation in which the pointer moves within the menu area R, and the user's operation when it is predicted that the user does not intend to operate the menu is an operation in which the pointer moves relatively farther away from the menu area R or an operation in which the pointer stops within the menu area R without moving.

[0092] In contrast, the second embodiment differs from the first embodiment in that the user's operations when it is predicted that the user intends to operate the menu include an operation to add an annotation, and the user's operations when it is predicted that the user does not intend to operate the menu include an operation to not add an annotation. The following description will focus on the differences from the first embodiment, and will omit a description of the points in common with the first embodiment. In the description of the second embodiment, the same parts as in the first embodiment will be described with the same reference numerals.

[0093] 12 is a diagram illustrating another example of a user operation when it is predicted that the user intends to operate a menu. AN in the diagram represents an annotation added by the user to a medical image. As shown in the diagram, when the user adds the annotation AN, the operation detection function 233 predicts that the operation indicates that the user intends to operate the menu. In response to this, the transmission control function 232 switches the transmission mode to the low-latency mode while the user is adding the annotation AN, that is, while it is continuously predicted that the user intends to operate the menu.

[0094] On the other hand, if the user does not add an annotation AN, the operation detection function 233 predicts that the user has no intention to operate the menu. In response to this, the transmission control function 232 switches the transmission mode to the high-image-quality mode while the user is not adding an annotation AN, that is, while it continues to be predicted that the user has no intention to operate the menu. In this way, by reducing the delay when adding annotations, the user can add annotations comfortably while observing the medical image.

[0095] Furthermore, as shown in Figure 7 above, when medical images from multiple medical image diagnostic devices 100 are transmitted to the terminal device 300 as medical video, the transmission control function 232 may determine the transmission mode of the medical image operated by the user among the multiple medical images to be the low latency mode, and may determine the transmission mode of the medical image not operated by the user among the multiple medical images to be the high image quality mode.

[0096] For example, suppose that medical images IMG1 to IMG6 are transmitted to the terminal device 300 and the user operates medical image IMG4. In this case, the transmission control function 232 determines the transmission mode for medical image IMG4 to be the low latency mode. Furthermore, the transmission control function 232 determines the transmission modes for medical images IMG1 to IMG3 and IMG5 to IMG6, excluding medical image IMG4, to be the high image quality mode. By controlling the transmission mode for each medical image IMG in this way, the user can operate more comfortably.

[0097] According to the second embodiment described above, the processing circuitry 230 of the server 200 controls the transmission mode of medical information transmitted to the terminal device 300 depending on whether an annotation is added or not. Furthermore, when a plurality of pieces of medical information are transmitted to the terminal device 300, the processing circuitry 230 determines the transmission mode of the medical information operated by the user among the plurality of pieces of medical information to be the low latency mode, and determines the transmission mode of the medical information not operated by the user among the plurality of pieces of medical information to be the high image quality mode. Such control of the transmission mode allows the user to more comfortably observe and operate medical images on the terminal device 300.

[0098] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0099] 1... medical information processing system, 100... medical image diagnostic apparatus (ultrasound diagnostic apparatus), 110... communication interface, 120... input interface, 130... output interface, 140... ultrasonic probe, 150... processing circuit, 160... memory, 200... server, 210... communication interface, 220... memory, 230... processing circuit, 231... acquisition function, 232... transmission control function, 233... operation detection function, 234... output control function, 300... terminal device, 310... communication interface, 320... input interface, 330... output interface, 340... processing circuit, 350... memory

Claims

1. an acquisition unit that acquires a user's operation history for medical information generated by a medical image diagnostic device, the medical information being transmitted to and displayed on a terminal device that is remote from the medical image diagnostic device; a transmission control unit that controls a transmission mode of medical information transmitted from the medical image diagnostic apparatus to the terminal device based on the operation history; A medical information processing device comprising:

2. the medical information includes at least a medical image generated by the medical image diagnostic device and a menu for allowing the user to specify conditions for generating or displaying the medical image; the transmission control unit controls the transmission mode based on the operation history of the user on the menu. The medical information processing device according to claim 1 .

3. The user's operation history includes a movement trajectory of a pointer; The transmission control unit When the pointer approaches the menu relatively, the transmission mode is determined to be a low-delay mode in which the medical image is displayed on the terminal device with a delay less than a threshold; When the pointer moves relatively farther away from the menu, the transmission mode is determined to be a high-image-quality mode in which the image quality of the medical image displayed on the terminal device is higher than that of the low-latency mode. The medical information processing device according to claim 2 .

4. The user's operation history includes a movement trajectory of a pointer; The transmission control unit When the pointer moves within the menu, the transmission mode is determined to be a low-latency mode in which the medical image is displayed on the terminal device with a latency less than a threshold; When the pointer stops within the menu, the transmission mode is determined to be a high-image-quality mode in which the image quality of the medical image displayed on the terminal device is higher than that of the low-latency mode. The medical information processing device according to claim 2 or 3.

5. the operation history of the user includes an operation of adding an annotation to the medical image; The transmission control unit determining the transmission mode to a low-latency mode in which the medical image is displayed on the terminal device with a latency less than a threshold while the annotation is being added; determining the transmission mode to a high-image-quality mode in which the image quality of the medical image displayed on the terminal device is higher than that of the low-latency mode while the annotation is not added; The medical information processing device according to claim 2 or 3.

6. the medical information of each of the plurality of medical image diagnostic devices is displayed on the terminal device; The transmission control unit determining the transmission mode of first medical information, which is the medical information operated by the user, among the medical information of each of the plurality of medical image diagnostic devices, to a low latency mode, which is a mode in which the first medical image is displayed on the terminal device with a latency less than a threshold; determining the transmission mode of second medical information, which is the medical information not operated by the user, among the medical information of each of the plurality of medical image diagnostic devices, to a high image quality mode in which the image quality of the second medical information displayed on the terminal device is higher than that of the low latency mode; The medical information processing device according to claim 2 or 3.

7. A medical information processing method using a computer, comprising: Acquiring a user's operation history for medical information generated by a medical image diagnostic device, the medical information being transmitted to and displayed on a terminal device remote from the medical image diagnostic device; Controlling a transmission mode of medical information transmitted from the medical image diagnostic apparatus to the terminal device based on the operation history; A medical information processing method including:

8. A program to be executed by a computer, Acquiring a user's operation history for medical information generated by a medical image diagnostic device, the medical information being transmitted to and displayed on a terminal device remote from the medical image diagnostic device; Controlling a transmission mode of medical information transmitted from the medical image diagnostic apparatus to the terminal device based on the operation history; Programs including.

Images