Medical image diagnostic system, and operation method of medical image diagnostic system
The medical image diagnosis system enables subjects to control video content and environmental lighting during examinations, addressing anxiety and stress associated with long and confined medical imaging procedures.
Patent Information
- Application Number
- JP2023081130
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-16
- Publication Date
- 2024-11-28
AI Technical Summary
Existing medical imaging technologies, such as MRI and X-ray CT, cause anxiety and stress due to long examination times and confined spaces, with subjects having limited control over the examination environment.
A medical image diagnosis system that allows subjects to control video content and environmental lighting during examinations using various detection methods, including cameras and microphones, to enhance relaxation and reduce stress.
Subjects can actively engage with video content and environmental adjustments, reducing anxiety and stress during medical imaging procedures.
Smart Images

Figure 2024165197000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a medical image diagnostic system and an operating method of a medical image diagnostic system, and more particularly to a technique for reducing anxiety and stress of a subject undergoing an examination using an image diagnostic apparatus. [Background technology]
[0002] The time it takes to examine a subject using imaging diagnostic equipment such as a magnetic resonance imaging (MRI) device or an X-ray computed tomography (CT) device is relatively long, with an MRI device taking around 20 to 30 minutes, an X-ray CT device taking around 5 to 10 minutes for a simple test, and around 5 to 20 minutes for a contrast test.
[0003] In addition, imaging using MRI devices and other equipment is performed in a cylindrical imaging space called a "bore," and because bodily movement during the examination can cause image artifacts, it is necessary to minimize bodily movement, including breathing. Being examined for long periods of time in a closed space called a "bore," while suppressing bodily movement, is stressful for the subject.
[0004] In view of this, a technique has been proposed in the past in which an image that can be viewed by the subject inside the bore is projected to relax the subject during the examination (Patent Documents 1 and 2). [Prior art documents] [Patent documents]
[0005] [Patent Document 1] Special Publication No. 2020-535931 [Patent Document 2] JP 2017-080299 A Summary of the Invention [Problem to be solved by the invention]
[0006] However, neither Patent Documents 1 nor 2 discloses a technique for allowing a subject to control the image during the examination.
[0007] Although Patent Document 1 states that "the entertainment video content can be selected by the patient" (Claim 37 of Patent Document 1), it does not state anything about a specific configuration.
[0008] Patent Document 2 also describes a projector that projects a first image onto a screen inserted into the bore and a second image onto the inner wall of the bore, the second image being trimmed according to the first image. Patent Document 2 also describes that a projection image projected by the projector is generated in response to an instruction from an operator so that at least one of the hue, brightness, and pattern is different between the first and second images, but does not describe an instruction by a subject under examination to change at least one of the hue, brightness, and pattern.
[0009] The present invention has been made in consideration of the above circumstances, and has an object to provide a medical image diagnostic system and an operating method of a medical image diagnostic system that enable a subject to control images during an examination using an image diagnostic device, thereby reducing the anxiety and stress of the subject during the examination. [Means for solving the problem]
[0010] The invention according to a first aspect is a medical image diagnostic system comprising: an image display that displays an image in a manner visible to a subject while the subject is being examined by an image diagnostic device; an operation detector that detects operations related to the image displayed on the image display from movements of the subject during the examination; a processor; and a memory that stores a program to be executed by the processor, wherein the processor receives an operation instruction indicating an operation related to the image detected by the operation detector, and performs control related to the image displayed by the image display based on the content of the received operation instruction.
[0011] According to the first aspect of the present invention, the image viewed by the subject can be controlled based on the movements of the subject during the examination, and the image that relaxes the subject can be utilized more effectively.
[0012] In the medical image diagnostic system according to the second aspect of the present invention, in the first aspect, it is preferable that the operation related to the video includes at least one of selection, switching of video content to be displayed on the video display, fast-forwarding, rewinding, pausing, playing and adjusting the volume of the video displayed on the video display.
[0013] In the medical image diagnostic system according to the third aspect of the present invention, in the first or second aspect, it is preferable that the operation detector includes a visible light camera, and detects an operation related to the image based on at least one of the movement of the subject's fingers or limbs, the movement of the eyes, the movement of the eyelids, and the movement of the lips detected from the image captured by the visible light camera.
[0014] In the medical image diagnostic system according to a fourth aspect of the present invention, in any one of the first to third aspects, the operation detector preferably includes an infrared camera and detects an operation related to the image based on the movement of the subject's fingers or limbs detected from the image captured by the infrared camera. In the case of an infrared camera, even if a towel blanket is draped over the subject to protect him / her from the cold, the movement of the fingertips, etc. can be captured through the towel blanket by using a towel blanket made of a thin material that easily transmits infrared light.
[0015] In the medical image diagnostic system according to the fifth aspect of the present invention, in any one of the first to fourth aspects, it is preferable that the operation detector includes an operation device operated by the subject and detects an operation related to the image according to the operation content of the operation device. As the operation device, an emergency contact switch given to the subject before the start of the examination, or a remote control switch for operating the image, etc. can be used.
[0016] In the medical image diagnostic system according to a sixth aspect of the present invention, in any one of the first to fifth aspects, it is preferable that the operation detector includes a first microphone, and detects an operation related to the image by recognizing a voice uttered by the subject detected by the first microphone.
[0017] In the medical image diagnostic system according to a seventh aspect of the present invention, in any one of the first to sixth aspects, the image diagnostic device images a preset examination site among a plurality of examination sites including the head and upper limbs of a subject, and the operation detector includes a plurality of detectors of different types, and it is preferable that a detector having a lower influence on the imaging of the examination site than other detectors is selected from the plurality of detectors, in order to suppress the occurrence of artifacts in the image captured by the image diagnostic device.
[0018] In a medical image diagnostic system according to an eighth aspect of the present invention, in the seventh aspect, it is preferable that the processor receives an operation instruction from a detector among the multiple detectors that has a lower influence on imaging of the examination site than other detectors, depending on the examination site of the imaging diagnostic device.
[0019] In the medical image diagnostic system according to a ninth aspect of the present invention, in any one of the first to eighth aspects, it is preferable that the processor displays an operation window that supports an operation related to the image by the operation detector on the image display device, so that the subject can perform detailed operations by the operation detector while looking at the operation window.
[0020] A medical image diagnostic system according to a tenth aspect of the present invention, in any one of the first to ninth aspects, preferably includes an ambient light adjuster for adjusting ambient light in a bore of the image diagnostic device, and the processor receives an operation instruction for adjusting the ambient light from the operation detector and controls the ambient light adjuster, in order to adjust the ambient light to the subject's preference or to make it easier to view an image displayed in the bore.
[0021] A medical image diagnostic system according to an eleventh aspect of the present invention, in any one of the first to ninth aspects, preferably includes an environmental light adjuster for adjusting environmental light in the bore of the image diagnostic device, and the processor controls the environmental light adjuster according to the content of the image displayed on the image display device, in order to automatically adjust the image displayed in the bore so that it is easy to see.
[0022] A medical image diagnostic system according to a twelfth aspect of the present invention, in any one of the first to ninth aspects, preferably includes an ambient light adjuster for adjusting ambient light within a bore of the imaging diagnostic device, and the processor preferably controls the ambient light adjuster in synchronization with a video scene displayed on the video display.
[0023] A thirteenth aspect of the present invention relates to a medical image diagnostic system in any one of the tenth to twelfth aspects, wherein the environmental light adjuster adjusts at least one of hue, lightness, and saturation.
[0024] A medical image diagnostic system according to a 14th aspect of the present invention is any one of the 1st to 13th aspects, wherein the image display is a projector that projects onto a screen visible to the subject, a head-up display, a head-mounted display, a liquid crystal display, or an organic electroluminescence display.
[0025] A medical image diagnostic system according to a fifteenth aspect of the present invention is any one of the first to fourteenth aspects, in which the image diagnostic device includes a magnetic resonance imaging device, an X-ray CT device, a PET device, a radiation therapy device, or a particle beam therapy device. The present invention is suitable for an image diagnostic device in which the examination time and treatment time are relatively long.
[0026] A 16th aspect of the present invention is a method for operating a medical image diagnostic system including an image display that displays an image in a manner visible to a subject during an examination of the subject by an image diagnostic device, an operation detector that detects an operation related to the image displayed on the image display, and a processor, the method including the steps of: the operation detector detecting an operation related to the image based on a movement of the subject during the examination; the processor receiving, from the operation detector, an operation instruction indicating an operation related to the image; and the processor controlling the image displayed by the image display based on the content of the received operation instruction.
[0027] In the operating method of the medical image diagnostic system according to the 17th aspect of the present invention, in the 16th aspect, it is preferable that the operation related to the video includes at least one of selection, switching of video content to be displayed on the video display, fast-forwarding, rewinding, pausing, playing and adjusting the volume of the video displayed on the video display. Effect of the Invention
[0028] According to the present invention, a subject undergoing an examination using an imaging diagnostic apparatus can control the images he or she views, thereby reducing the subject's anxiety and stress during the examination. [Brief description of the drawings]
[0029] [Figure 1] FIG. 1 is a perspective view showing an example of an image diagnostic apparatus to which a medical image diagnostic system according to the present invention is applied. [Diagram 2] FIG. 2 is a schematic cross-sectional view of the MRI apparatus shown in FIG. [Diagram 3] FIG. 3 is a diagram showing how the subject's body is turned relative to the MRI apparatus. [Figure 4] FIG. 4 shows the inside of an operation room where the MRI apparatus is operated. [Diagram 5] FIG. 5 is a configuration diagram of a medical image diagnostic system according to the present invention. [Figure 6]FIG. 6 is a block diagram showing an embodiment of the hardware of a signal processing device which is a main device constituting a medical image diagnostic system. [Figure 7] FIG. 7 is a diagram showing an example of an image when an operation related to the image is detected based on the movement of the subject's finger and the operation related to the image is executed. [Figure 8] FIG. 8 is a diagram showing an example of an image when an operation related to the image is detected based on the movement of the subject's hand and the operation related to the image is executed. [Figure 9] FIG. 9 is a diagram showing how an image including a subject's hand is captured. [Figure 10] FIG. 10 is a diagram showing an example of an image when an operation related to the image is detected based on the movement of the subject's line of sight and the operation related to the image is executed. [Figure 11] FIG. 11 is a diagram showing another example of an image when an operation related to the image is detected based on the subject's line of sight movement or the like and the operation related to the image is executed. [Figure 12] FIG. 12 is a conceptual diagram showing an embodiment of the utterance recognition unit. [Figure 13] FIG. 13 is a perspective view of a main part of an example of an operating device operated by a subject. [Figure 14] FIG. 14 is a perspective view showing another example of an operating device operated by a subject. [Figure 15] FIG. 15 is a conceptual diagram showing another embodiment of the control related to the video. [Figure 16] FIG. 16 is a diagram showing another embodiment of the image display device that displays images in a manner that can be visually recognized by the subject during an examination using an MRI apparatus. [Figure 17] FIG. 17 is a table showing the relationship between the inspection site, the detection method, the degree of influence, and the selection of the detection method. [Figure 18] FIG. 18 is a flow chart illustrating an embodiment of a method of operating a medical image diagnostic system according to the present invention. [Figure 19] FIG. 19 is a flow chart showing another embodiment of the method of operating the medical image diagnostic system according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0030] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of a medical image diagnostic system and an operating method of the medical image diagnostic system according to the present invention will be described with reference to the accompanying drawings.
[0031] FIG. 1 is a perspective view showing an example of an image diagnostic apparatus to which a medical image diagnostic system according to the present invention is applied.
[0032] The imaging diagnostic device shown in FIG. 1 is a magnetic resonance imaging device (MRI device) 10, and includes a gantry 11 and a bed 13 with a tabletop 13A arranged on the front side of a bore 12, which is a cylindrical imaging space provided in the gantry 11.
[0033] The image diagnostic device in this example is an MRI device 10, but the present invention is not limited to this and includes an X-ray CT device, a PET (Positron Emission Tomography) device, a radiation therapy device, or a particle beam therapy device. The present invention is suitable for devices with relatively long examination and treatment times.
[0034] FIG. 2 is a schematic cross-sectional view of the MRI apparatus shown in FIG.
[0035] 2, an MRI magnet 10A and various other coils are arranged in a gantry 11, and a subject 5 lying on a top board 13A is moved to a bore 12 so that an examination site of the subject 5 is located at the center of a static magnetic field in the bore 12 by moving the top board 13A to a bore 12. An RF (Radio Frequency) coil 10B for receiving FID (Free Induction Decay) from the human body is arranged at the examination site of the subject 5. In this example, the abdomen of the subject 5 is the examination site, and the RF coil 10B is arranged on the abdomen.
[0036] Visible light cameras 15 (15A, 15B) and first microphones 16 (16A, 16B) are disposed at the top of both ends of bore 12. In addition, in FIG. 2, camera 15A and first microphone 16A are disposed at the top of the left end of bore 12, and camera 15B and first microphone 16B are disposed at the top of the right end of bore 12.
[0037] The two cameras 15A, 15B each capture an image of the subject 5 and output a moving image (video), and at least one of the two cameras 15A, 15B captures an image including the face region of the subject 5.
[0038] The two first microphones 16A and 16B each detect the voice emitted by the subject 5. The first microphones 16A and 16B are preferably non-magnetic microphones that use piezoelectric ceramics, which are non-magnetic materials, in their sound receiving portions.
[0039] A projector 17 is disposed at one end (the left end in FIG. 2) of the top board 13A, and the projector 17 projects an image onto a screen 17A disposed on the ceiling inside the bore 12. This projector 17 functions as an image display device that displays an image in a manner that is visible to the subject 5 during an examination using the MRI device 10, and the subject 5 can view the image projected onto the screen 17A during the examination. Note that the screen 17A may not be provided, and the ceiling inside the bore 12 may be used as the screen.
[0040] 2, 18 is a vital information measuring device that measures one or more of the heart rate, blood pressure, respiratory rate, body temperature, electrocardiogram, and blood oxygen saturation concentration of the subject 5. The vital information measuring device 18 is not limited to being placed on the back of the subject 5.
[0041] Specific vital information measuring devices 18 may include an ECG (electrocardiogram) sensor that measures electrocardiogram and heart rate, an SpO2 sensor that measures blood oxygen saturation, a bellows that detects respiratory rate, an optical camera capable of detecting the facial expression and body temperature of the subject 5, an infrared camera, a thermograph that measures body temperature, an NMR (Nuclear Magnetic Resonance) signal (the signal itself obtained by an MRI examination), an MRI image (an image of the NMR signal), other sensors (pressure sensors, optical sensors, ultrasonic sensors, etc.), and combinations of these.
[0042] FIG. 3 is a diagram showing how the subject's body is turned relative to the MRI apparatus.
[0043] FIG. 3(A) shows the case where the subject 5 enters the bore 12 head first (head first), and FIG. 3(B) shows the case where the subject 5 enters the bore 12 feet first (foot first).
[0044] Head-first or feet-first is selected depending on the examination site of the subject 5. In this example, cameras 15A, 15B and first microphones 16A, 16B are disposed at both ends of the bore 12 so that the facial region of the subject 5 can be photographed and the voice can be detected regardless of whether head-first or feet-first is selected.
[0045] In this example, the number of cameras 15 (15A, 15B) and first microphones 16 (16A, 16B) is two, but the number is not limited to two and may be one, or three or more. Also, the camera 15 and the first microphone 16 are not limited to being placed in the MRI device 10, and may be placed near the MRI device 10.
[0046] FIG. 4 shows the inside of an operation room where the MRI apparatus is operated.
[0047] As shown in FIG. 4, the operation room 2 is equipped with a signal processing device 20 that controls the MRI device 10 installed in the examination room, processes the NMR signals output from the MRI device 10, and generates MRI images.
[0048] The signal processing device 20 can be configured using a computer. The computer applied to the signal processing device 20 may be a personal computer or a workstation. An operator 28 can control the operation of the MRI apparatus 10 through the signal processing device 20 by using an operation unit such as a keyboard and a mouse.
[0049] The signal processing device 20 of this embodiment also has some of the functions of the medical image diagnostic system according to the present invention, the details of which will be described later.
[0050] The signal processing device 20 includes a display 24 for displaying MRI images and the like, a microphone (second microphone) 25 for detecting the voice of an operator 28, and a speaker 26 for generating voices and the like uttered by the subject 5 during an examination.
[0051] 4, 2A is a window through which an operator 28 can observe the state of the examination room. 29 is a display that displays images captured by the cameras 15A and 15B, vital sign information, and the like.
[0052] [Medical image diagnostic system configuration] FIG. 5 is a configuration diagram of a medical image diagnostic system according to the present invention.
[0053] The medical image diagnostic system shown in FIG. 5 is configured by connecting a camera 15, a first microphone 16, a projector 17, a vital information measuring device 18, an operating device 19 described later, an imaging section (not shown) including a pulse sequence section of the MRI device 10, and a signal processing device 20 in an operating room 2, which are all arranged in the MRI device 10, via a network 50 such as a LAN (Local area network).
[0054] FIG. 6 is a block diagram showing an embodiment of the hardware of a signal processing device which is a main device constituting a medical image diagnostic system.
[0055] The signal processing device 20 shown in FIG. 6 is configured by a computer or the like as described above, and includes a processor 21, a memory 22, an input / output interface 23, a display unit 24, a speaker 26, an operation unit 27, and the like.
[0056] The processor 21 is composed of a CPU (Central Processing Unit) etc., and controls each part of the signal processing device 20 and various devices arranged in the MRI device 10, and realizes various functions described later by executing a signal processing program stored in the memory 22.
[0057] The memory 22 is one or more memories including a flash memory, a read-only memory (ROM), a random access memory (RAM), a hard disk drive, etc. The flash memory, the ROM, and the hard disk drive are non-volatile memories that store an operation system, a signal processing program that causes the processor 21 to function as the signal processing device 20, various programs including a machine learning model, and a plurality of video contents projected by the projector 17, etc.
[0058] The RAM functions as a working area for processing by the processor 21, and also temporarily stores information processing programs and the like stored in the non-volatile memory. Note that the processor 21 may incorporate a part of the memory 22 (RAM).
[0059] The input / output interface 23 includes a communication unit connectable to the network 50, and a connection unit connectable to an external device. As the connection unit connectable to an external device, a Universal Serial Bus (USB), a High-Definition Multimedia Interface (HDMI) (HDMI is a registered trademark), etc. can be applied.
[0060] The processor 21 communicates with various devices provided in the MRI apparatus 10 via the input / output interface 23 and the network 50, thereby transmitting and receiving necessary information.
[0061] The display 24 is used as part of a GUI (Graphical User Interface) for receiving characters indicating what the subject 5 says during the examination, MRI images, personal information of the subject 5, and input from the operation unit 27.
[0062] The microphone 25 detects the voice uttered by the operator 28 and outputs an audio signal indicating the detected voice to the signal processing device 20.
[0063] The speaker 26 receives an audio signal from the signal processing device 20 and generates a sound or the like indicating the contents of the statements made by the subject 5 during the examination.
[0064] The operation unit 27 includes a mouse, a keyboard, etc., and functions as a part of a GUI that uses the display operation window of the display 24 and receives input from the operator 28.
[0065] [Medical imaging diagnostic system functions] The processor 21 of the signal processing device 20 can cause the projector 17 to project an image for relaxing the subject 5 onto the screen 17A.
[0066] That is, the processor 21 transmits video content selected from the plurality of video contents stored in the memory 22 to the projector 17 via the input / output interface 23 and the network 50, and the projector 17 projects an image based on the received video content onto the screen 17A in the bore 12. The plurality of video contents include environmental images depicting nature such as the sky, sea, and mountains, images of plants and animals or tourist spots, images such as animations for children, or images showing books.
[0067] During the examination, the subject 5 can relax by watching the image projected on the screen 17A.
[0068] In particular, the medical image diagnostic system according to the present invention is configured to allow the subject 5 to control the images during the examination. Here, "control of the images" includes selection and switching of image content, fast forwarding, rewinding, pausing, playing, adjusting the volume, turning pages of a book, etc.
[0069] <First embodiment of operation detector for detecting operations related to video> The operation detector of the first embodiment includes a visible light camera (camera 15) that captures an image of the subject 5 during the examination, and detects operations related to the image based on the movements of the subject's 5 fingers or limbs from the video (image) captured by the camera 15.
[0070] FIG. 7 is a diagram showing an example of an image when an operation related to the image is detected based on the movement of the subject's finger and the operation related to the image is executed.
[0071] In FIG. 7, an operation window 171 for assisting an operation related to the image is superimposed on an image 170 for relaxing the subject 5 during the examination.
[0072] When the processor 21 detects a finger movement (wake-up movement) of the subject 5 from the image captured by the camera 15, it is preferable to pop up and display an operation window 171 having icon buttons for instructing fast-forwarding, rewinding, pausing, and playing the image. The subject 5 moves the icon 171A (hand-shaped icon 171A in this example) by moving his / her finger or hand while looking at the operation window 171. That is, the processor 21 moves the icon 171A according to the hand or finger movement of the subject 5 from the image captured by the camera 15.
[0073] The subject 5 moves the index finger of icon 171A onto one of the icon buttons for fast-forwarding, rewinding, pausing, or playing the video, and then, for example, taps the index finger to instruct fast-forwarding, rewinding, pausing, or playing the video. The processor 21 detects the above-mentioned movement of the hand or finger of the subject 5 from the video captured by the camera 15, and thereby fast-forwards, rewinds, pauses, or plays the video.
[0074] It is preferable that the processor 21 erases the operation window 171 when the movement of the hand or finger of the subject 5 is not detected for a certain period of time or more after the operation window 171 is popped up, or when playback is instructed. The operation window 171 may be constantly displayed on the image 170. In this case, it is preferable that the operation window for supporting the operation related to the image is displayed at the edge of the image so as not to interfere with the image 170.
[0075] FIG. 8 is a diagram showing an example of an image when an operation related to the image is detected based on the movement of the subject's hand and the operation related to the image is executed, and particularly shows a case where the image is switched.
[0076] In FIG. 8, an operation window 172 (operation window for switching between images) for supporting operations related to the image is superimposed on an image 170 for relaxing the subject 5 during the examination.
[0077] By rotating the hand in a counterclockwise or clockwise direction, the subject 5 during the examination can switch to the next or previous image in place of the currently displayed image 170. That is, the processor 21 detects the hand movement (counterclockwise or clockwise rotation movement) of the subject 5 from the image captured by the camera 15 in the same manner as described above, and switches and displays the next or previous image in place of the currently displayed image 170.
[0078] In addition, the video is not limited to being switched, and for example, a list of video contents may be displayed, and the subject 5 may select a desired video content by a finger or hand movement. When the processor 21 receives an instruction to select a video content by a finger or hand movement of the subject 5, the processor 21 displays the received video.
[0079] FIG. 9 is a diagram showing how an image including a subject's hand is captured.
[0080] 9, a towel blanket 152 may be placed over the subject 5 during the examination to protect the subject from the cold. In this case, the camera 15, which is a visible light camera, cannot capture the hands of the subject 5 under the towel blanket 152.
[0081] Therefore, it is preferable that the towel blanket 152 has an opening 152A through which the subject 5 can put out his / her hands.
[0082] On the other hand, an infrared camera can be used instead of the camera 15. In the case of an infrared camera, even if a towel blanket is draped over the subject 5, fingertip movements and the like can be captured through the towel blanket by using a towel blanket made of a thin material that easily transmits infrared light.
[0083] The camera 15 of the operation detector of the first embodiment may be set to selectively capture the fingers of either one of both hands of the subject 5. For example, in order to suppress the occurrence of artifacts in MRI images, it is preferable to capture the fingers farthest from the examination site or the dominant hand of the subject 5. In addition, the movement of the fingers is not limited, and for example, the movement of the toes may be captured to detect the operation related to the image.
[0084] <Second embodiment of operation detector for detecting operations related to video> The operation detector of the second embodiment includes a camera 15 that images the subject 5 during the examination, and detects operations related to the image based on the movement of the subject's 5 line of sight from the image captured by the camera 15 (particularly the image of the subject's 5 pupil area).
[0085] FIG. 10 is a diagram showing an example of an image when an operation related to the image is detected based on the movement of the subject's line of sight and the operation related to the image is executed, and shows an open book.
[0086] The processor 21 detects the movement of the pupils of the subject 5 from the image of the pupil region of the subject 5 captured by the camera 15. The arrows in Fig. 10 diagrammatically show the trajectory of the gaze direction associated with the movement of the pupils of the subject 5.
[0087] When the processor 21 detects from the movement of the pupils of the subject 5 that the line of sight has moved to the end of the page of the currently open book, it turns the page of the image of the book.
[0088] FIG. 11 is a diagram showing another example of an image when an operation related to the image is detected based on the subject's line of sight movement or the like and the operation related to the image is executed.
[0089] In the example shown in FIG. 11, an operation window 174 for assisting in operations related to an image 170 for relaxing the subject 5 during an examination is superimposed on the image 170.
[0090] When the processor 21 detects the movement (wake-up movement) of the eyes (eyelids) of the subject 5 from the image captured by the camera 15, it is preferable to pop up and display the operation window 174. As the wake-up movement in this case, for example, the movement of "blinking" twice in succession can be applied.
[0091] Furthermore, the processor 21 displays an operation window 174 including icon buttons for fast-forwarding, rewinding, pausing, and playing the video, and preferably further displays an arrow icon 174A indicating the gaze position of the subject 5.
[0092] The processor 21 moves the arrow icon 174A on the operation window 174 based on the movement of the pupils of the subject 5.
[0093] When subject 5 during the examination wishes to give instructions to fast-forward, rewind, pause, or play the video, for example, he or she moves arrow icon 174A indicating the gaze position to the position of the icon button indicating fast-forward, rewind, pause, or play of the video and stops it for a certain period of time (several seconds) or more (stares at it for a certain period of time or more).
[0094] The processor 21 accepts the instruction indicated by the icon button on which the arrow icon 174A is stopped for a certain period of time or longer as an operation related to the video, and controls the video to be displayed.
[0095] <Third embodiment of operation detector for detecting operations related to video> The operation detector of the third embodiment includes a camera 15 that images the subject 5 during the examination, and detects operations related to the image based on the movement of the subject 5's lips (so-called "lip-syncing") from the image captured by the camera 15 (particularly the image of the facial area of the subject 5).
[0096] The processor 21 functions as a statement recognition section that recognizes the content of a statement made by the subject 5 based on the movement of the subject's 5 lips included in the image of the subject's 5 face region.
[0097] FIG. 12 is a conceptual diagram showing an embodiment of the utterance recognition unit.
[0098] As shown in FIG. 12, the processor 21 functions as a utterance recognition section 21A.
[0099] The utterance recognition unit 21A is a lip reading part that reads characters from the movement of the lips of the subject 5, and in this example, a trained machine learning model is used, and an image including the facial area of the subject 5 captured by the camera 15 is input.
[0100] The processor 21 can read and use a trained machine learning model stored in advance in the memory 22. As the machine learning model, various learning models such as an end-to-end deep learning network, a convolutional neural network, and a support vector machine can be used.
[0101] The utterance recognition unit 21A in this example outputs multiple characters (a string of characters indicating the content of the utterance) uttered by the subject 5 one by one from an input video showing the lip movement of the subject 5, but it is preferable that the unit also has a function of performing natural language processing on the string of characters output from the utterance recognition unit 21A to determine whether the string of characters is a meaningful utterance, and, if there is an error as a result of the natural language processing on the string of characters, correcting it to make it the correct utterance content.
[0102] Furthermore, since the content of a statement indicating an operation related to a video is determined according to the type of operation related to the video, the statement recognition unit 21A only needs to be able to correctly recognize the type of operation related to the video, and therefore the recognition accuracy can be made high.
[0103] Furthermore, the utterance recognition unit 21A may simultaneously input the video including the facial region of the subject 5 captured by the camera 15 and the voice of the subject 5 detected by the first microphone 16, and recognize the content of the utterance of the subject. In this case, the machine learning model applied to the utterance recognition unit preferably includes a learning model that recognizes the content of the utterance from the video including the facial region of the subject 5 (movement of the lips of the subject 5) and a learning model that recognizes the content of the utterance from the voice of the subject 5, and both of which are trained to show the same utterance content.
[0104] Processor 21 controls the image projected from projector 17 based on the recognition result recognized by utterance recognition section 21A.
[0105] <Fourth embodiment of operation detector for detecting operations related to video> The operation detector of the fourth embodiment includes an operation device operated by the subject 5 during the examination, and detects an operation related to the image according to the operation content of the operation device.
[0106] FIG. 13 is a perspective view of a main part of an example of an operating device operated by a subject.
[0107] The operating device 19 shown in FIG. 13 is, for example, a rubber bulb switch 190 for emergency contact.
[0108] The rubber bulb switch 190 is carried by the subject 5 for emergency contact, and during the test, the subject 5 presses (tightly grips) the rubber bulb switch 190 when making an emergency contact. The pressure change caused by the operation of the rubber bulb switch 190 is detected by a pressure switch (not shown) via a tube 190B.
[0109] In this example, the rubber bulb switch 190 also serves as an operation detector that detects an operation related to the image.
[0110] For example, when using the rubber ball switch 190 for normal emergency contact, the subject 5 presses the rubber ball switch 190 for a long time, and when using it for an operation related to an image, the subject presses the rubber ball switch 190 according to a pressing pattern corresponding to the operation related to the image. For example, when increasing the volume of the sound generated along with the image, the subject presses the rubber ball switch 190 once briefly, and when decreasing the volume, the subject presses the rubber ball switch 190 twice briefly. Note that a plurality of pressing patterns are prepared according to the type of operation related to the image.
[0111] When the processor 21 detects that the rubber bulb switch 190 has been pressed and held, the processor 21 performs processing such as stopping the MRI apparatus 10.
[0112] On the other hand, when processor 21 detects that rubber ball switch 190 has been pressed with a pressing pattern corresponding to the type of operation related to the image, processor 21 performs control related to the image assigned to that pressing pattern.
[0113] FIG. 14 is a perspective view showing another example of an operating device operated by a subject.
[0114] The operating device 19 shown in FIG. 14 is a push button switch 192 having push button portions 192A and 192B made up of two air balloons.
[0115] Push button switch 192 has two tubes 192C whose internal pressure changes in response to pressing of push button portion 192A or 192B, and the pressure change within tube 192C is detected by a pressure sensor (not shown).
[0116] For example, when processor 21 detects that push button section 192A has been pressed, it increases the volume of the sound generated along with the video, and when processor 21 detects that push button section 192B has been pressed, it decreases the volume.
[0117] Note that the instruction related to the image by operating the push button switch 192 is not limited to volume adjustment, and may be an instruction related to other images. In addition, the push button switch 192 in this example is placed on the top board 13A of the bed 13, and preferably has an anti-slip part 192D made of a rubber material or having a special protrusion so as not to slip on the top board 13A.
[0118] Furthermore, the operation unit is not limited to the rubber bulb switch 190 and the push button switch 192 shown in Figures 13 and 14, but any operation unit that outputs instructions related to the image operated by the subject 5 may be used.
[0119] <Fifth embodiment of operation detector for detecting operations related to video> The operation detector of the fifth embodiment includes a microphone (first microphone) 16 that detects the voice emitted by the subject 5 during the examination, and recognizes the voice emitted by the subject 5 detected by the first microphone 16 to detect operations related to the image.
[0120] The processor 21 has a function as a voice recognizer that converts the voice detected by the first microphone 16 into text information, and performs control related to an image specified by the text information based on the text information recognized by the voice recognizer. The voice recognizer may be configured with a voice recognition AI (Artificial Intelligence) that recognizes text from the voice detected by the first microphone 16. When the voice detected by the first microphone 16 is input to the voice recognition AI, it is preferable to reduce noise generated by the MRI apparatus 10 in advance using a filter or the like.
[0121] <Another embodiment of video control> FIG. 15 is a conceptual diagram showing another embodiment of image control, and shows a case where the ambient light in the bore in which the image is displayed is adjusted.
[0122] The medical image diagnostic system includes an ambient light adjuster that adjusts ambient light in the bore 12 of the MRI apparatus 10. The ambient light adjuster is a projector different from the projector 17, and can be configured by a projector that projects ambient light onto the entire inner wall surface of the bore 12, or a lighting device that illuminates the entire inner wall surface of the bore 12.
[0123] When the processor 21 receives an operation instruction to adjust the ambient light from the operation detector, the processor 21 controls the ambient light adjuster to adjust the ambient light within the bore 12 .
[0124] The operation detector can be, for example, any of the operation detectors of the first to fifth embodiments, and the subject 5 during the examination uses the operation detector to give instructions to change the ambient light in the bore 12 to the ambient light desired by the subject 5, and the processor 21 receives an operation instruction to adjust the ambient light from the operation detector and controls the ambient light adjuster.
[0125] This is to provide the subject 5 with an environmental light that suits his / her preference, or to make it easier to see the image displayed within the bore.
[0126] FIG. 15(A) is an image diagram when the brightness of the ambient light illuminating the inner wall surface inside the bore 12 is adjusted to be higher, FIG. 15(B) is an image diagram when the brightness of the ambient light illuminating the inner wall surface inside the bore 12 is adjusted to be lower, and FIG. 15(BC) is an image diagram when the color of the ambient light illuminating the inner wall surface inside the bore 12 is adjusted to be an incandescent color.
[0127] The ambient light adjuster adjusts at least one of the hue, brightness, and saturation of the ambient light within the bore 12, and may be, for example, neutral white, daylight, or warm white.
[0128] The processor 21 may also control the ambient light adjuster in accordance with the content of the image displayed in the bore 12. This is to automatically adjust the ambient light on the device side so that the image displayed in the bore 12 is easy to see.
[0129] Processor 21 may also control an ambient light regulator in synchronization with video scenes and audio displayed within bore 12 .
[0130] A thirteenth aspect of the present invention relates to a medical image diagnostic system in any one of the tenth to twelfth aspects, wherein the environmental light adjuster adjusts at least one of hue, lightness, and saturation.
[0131] <Another embodiment of the image display device> FIG. 16 is a diagram showing another embodiment of the image display device that displays images in a manner that can be visually recognized by the subject during an examination using an MRI apparatus.
[0132] The image display shown in Figure 16 is composed of a projector (not shown), a rear projection screen 60 onto which an image is projected from the projector, and a reflecting mirror 62 that allows the subject 5 lying on his / her back on the tabletop 13A to view the rear projection screen 60.
[0133] The rear projection screen 60 and the reflecting mirror 62 are disposed on a support member 63 attached to the top plate 13A.
[0134] When an image is projected from the projector onto the rear projection screen 60 , the subject 5 can view the image projected onto the rear projection screen 60 via the reflecting mirror 62 .
[0135] A camera for photographing the face area of the subject 5 can be attached to the outer periphery of the reflecting mirror 62. A speaker 16-1 for the subject can be embedded in a head holder 70 for holding the head of the subject 5 so as not to move. This speaker 16-1 can generate sound accompanying the video and the voice of the operator 28 when talking with the operator 28. The speaker 16-1 is preferably a non-magnetic speaker that uses piezoelectric ceramic, which is a non-magnetic material, in the sound-producing portion.
[0136] Other embodiments of the image display that displays images in a manner that can be viewed by the subject during examination using the MRI apparatus are not limited to the one shown in FIG. 16, and examples that can be used include a head-up display that displays a virtual image (aerial image), a head-mounted display that is worn on the subject's head, a liquid crystal display, or an organic EL display.
[0137] <Selection of operation detector> The MRI device 10 images a preset examination site among a plurality of examination sites including the head and upper limbs of a subject. In the example shown in Fig. 2, the abdomen of the subject 5 is the examination site, and the RF coil 10B is placed on the abdomen, but the FR coil is placed on the head when the examination site is the head, and is placed on the upper limbs when the examination site is the upper limbs. In addition, the position of the tabletop 13A is adjusted according to the position where the FR coil is placed.
[0138] As described in the first to fifth embodiments, the operation detector may detect an operation related to an image captured by a visible light camera (camera 15) based on the movements of the subject's 5 fingers and hands, detect an operation related to an image based on the gaze and eye (eyelid) movements, detect an operation related to an image based on lip movements, detect an operation related to an image based on an operation using an operating device (rubber bulb switch 190, push button switch 192), detect an operation related to an image based on sound detected by a microphone (first microphone 16), and so on.
[0139] Among these multiple detection methods, it is preferable to select a detector that has a lower effect on imaging of the examination site than other detectors, in order to suppress the occurrence of artifacts in MRI images.
[0140] That is, it is preferable that the operator 28 selects a detector having a lower degree of influence than other detectors from among a plurality of detection methods according to the examination site, and allows the subject 5 to use the selected detector.
[0141] In addition, the processor 21 can receive an operation instruction from a detector that has a lower influence on imaging of the examination site than other detectors among the multiple detectors according to the examination site. Since the processor 21 can obtain information on the examination site in advance, it can automatically determine from which detector the operation instruction should be received.
[0142] FIG. 17 is a table showing the relationship between the inspection site, the detection method, the degree of influence, and the selection of the detection method.
[0143] In Fig. 17, examples of the head and upper limbs are shown as the examination sites. As shown in Fig. 17, the degree of influence during imaging differs depending on the relationship between the examination site and the detection method, and accordingly, the selection of each detection method can be classified as "possible", "not possible", or "partially possible".
[0144] The parts to be examined are not limited to the head and upper limbs, but may include the trunk, lower limbs, etc. When the parts to be examined are the upper limbs, the method of detecting finger movements using a visible light camera is "not allowed," but when the part to be examined is the left upper limb, the method of detecting finger movements of the right upper limb using a visible light camera can be selected.
[0145] [Operation method of medical image diagnostic system] FIG. 18 is a flow chart illustrating an embodiment of a method of operating a medical image diagnostic system according to the present invention.
[0146] The processing of each step of the operating method of the medical image diagnostic system shown in FIG. 18 corresponds to the processing performed by the processor 21 of the signal processing device 20 shown in FIG.
[0147] 18, the processor 21 determines whether or not an operation instruction indicating an operation related to an image by the movement of the subject 5 during the examination has been input from the operation detector (step S10). This image is an image that is projected by the projector 17 into the bore 12 and viewed by the subject 5, and is an image for relaxing the subject 5 during the examination.
[0148] When an operation instruction indicating an operation related to the image is input (if "Yes"), the processor 21 determines whether the operation related to the image is a playback instruction or not (step S20). When a playback instruction is input, the processor 21 causes the image to be played back (step S30). That is, the processor 21 causes the projector 17 to project the image onto the screen 17A in the bore 12. Note that the image may be automatically played back when the MRI device 10 starts an examination.
[0149] If the input operation instruction is not a playback instruction, processor 21 subsequently determines whether or not a fast-forward instruction has been input (step S40). If a fast-forward instruction has been input, processor 21 fast-forwards the video (step S50). Processor 21 fast-forwards the video supplied to projector 17, and causes projector 17 to project the video being fast-forwarded.
[0150] If the input operation instruction is not a fast-forward instruction, processor 21 subsequently determines whether or not a rewind instruction has been input (step S60). If a rewind instruction has been input, processor 21 rewinds the video (step S70). Processor 21 rewinds the video to be supplied to projector 17, and causes projector 17 to project the video being rewound.
[0151] Furthermore, if the input operation instruction is not a rewind instruction, processor 21 subsequently determines whether or not a pause instruction has been input (step S80). If a pause instruction has been input, processor 21 pauses the video (step S90). Processor 21 pauses the video supplied to projector 17, and causes projector 17 to project the paused video.
[0152] In this way, the subject 5 undergoing the examination can send operation instructions to play, fast forward, rewind, or pause the image in accordance with his or her own movements to the processor 21 via the operation detector, and when the processor 21 receives an operation instruction to play, fast forward, rewind, or pause the image from the operation detector, it controls the image projected from the projector 17 based on the content of the operation instruction.
[0153] This allows the subject 5 undergoing an examination to select the scene he or she wishes to view from the video and have it played back.
[0154] Moreover, the subject 5 during the examination can switch to the image he or she wants to see, or select the image, and can also adjust the volume.
[0155] FIG. 19 is a flow chart showing another embodiment of the method of operating the medical image diagnostic system according to the present invention.
[0156] The processing of each step of the operating method of the medical image diagnostic system shown in FIG. 19 corresponds to the processing performed by the processor 21 of the signal processing device 20 shown in FIG.
[0157] 19, the processor 21 determines whether or not an operation instruction indicating an operation related to the image by the movement of the subject 5 during the examination has been input from the operation detector (step S100). If an operation instruction indicating an operation related to the image has been input (if "Yes"), the processor 21 transitions to step S20 shown in FIG. 18, and thereafter executes the process according to the flowchart shown in FIG. 18 described above.
[0158] In step S100, if no operation instruction indicating an operation related to the image is input (if "No"), when a certain period of time has elapsed without inputting an operation instruction, the processor 21 further acquires information related to changes in the physical condition of the subject 5 resulting from the examination (e.g., changes in physical condition such as faster heart rate) based on one or more vital information of the subject 5 measured by the vital information measuring device 18, including the heart rate, blood pressure, respiratory rate, body temperature, electrocardiogram, and blood oxygen saturation concentration, and determines whether it is appropriate to switch the image provided to the subject 5 (step S110).
[0159] When the processor 21 determines that a non-operation period has elapsed for a certain period of time or more and that it would be better to switch the image provided to the subject 5, the processor 21 proposes switching the image (step S120). This is to change the subject 5's mood by determining that the subject 5 is feeling anxious or stressed and switching the image.
[0160] The suggestion to switch the image can be made by superimposing text information of the suggested content on the image projected by the projector 17. For example, the suggestion is made by text information such as "If you want to change the image, please try operating XX." In addition, the suggestion is not limited to text information, but can be made by generating a voice (automatic voice) from the subject's speaker indicating the same operation content as the text information.
[0161] Thereafter, the processor 21 determines whether or not the subject 5 has input an instruction to switch the image in accordance with the suggestion to switch the image (step S130), and if it determines that an instruction to switch the image has been input, the processor 21 switches the image projected from the projector 17 to another image.
[0162] It is preferable that the processor 21 cancels the suggestion to switch the video when the subject 5 does not give an instruction to switch the video and when it is determined that a certain period of time has elapsed. Also, when the vital information of the subject 5 changes again, the processor 21 may suggest switching the video again.
[0163] [others] In this embodiment, for example, the hardware structure of a processing unit that executes various processes such as a CPU is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture such as an FPGA (Field Programmable Gate Array), and a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for executing specific processes such as an ASIC (Application Specific Integrated Circuit).
[0164] One processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with one processor. As an example of configuring multiple processing units with one processor, first, as represented by a computer such as a client or server, there is a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by a system on chip (SoC), there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.
[0165] Moreover, the hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.
[0166] Furthermore, the present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the spirit of the present invention. [Explanation of symbols]
[0167] 10...MRI device 11. Gantry 12…Boa 13. Bed 13A…Tabletop
Claims
1. an image display that displays an image in a manner that is visible to a subject during an examination of the subject by the image diagnostic device; an operation detector that detects an operation related to the image displayed on the image display device by a movement of the subject during the examination; A processor; A memory that stores a program to be executed by the processor; The processor, receiving an operation instruction indicating an operation related to the image detected by the operation detector; performing control related to the image displayed by the image display device based on the content of the received operation instruction; Medical imaging diagnostic system.
2. The operation related to the video includes at least one of selection and switching of video content to be displayed on the video display device, fast forwarding, rewinding, pausing, playing, and adjusting the volume of the video displayed on the video display device. The medical image diagnostic system according to claim 1 .
3. The operation detector includes a visible light camera, and detects an operation related to the image based on at least one of the subject's finger or limb movement, eye movement, eyelid movement, and lip movement detected from the image captured by the visible light camera. The medical image diagnostic system according to claim 1 .
4. The operation detector includes an infrared camera and detects an operation related to the image based on a movement of the subject's finger or hand or foot detected from an image captured by the infrared camera. The medical image diagnostic system according to claim 1 .
5. The operation detector includes an operation device operated by the subject, and detects an operation related to the image according to the operation content of the operation device. The medical image diagnostic system according to claim 1 .
6. The operation detector includes a first microphone and detects an operation related to the image by recognizing a voice uttered by the subject detected by the first microphone. The medical image diagnostic system according to claim 1 .
7. the imaging diagnostic apparatus captures an image of a preset examination site among a plurality of examination sites including the head and upper limbs of the subject; The operation detector includes a plurality of detectors of different types, and among the plurality of detectors, a detector that has a lower influence on imaging of the examination site than other detectors is selected. The medical image diagnostic system according to any one of claims 3 to 6.
8. the processor receives the operation instruction from a detector, among the plurality of detectors, that has a lower influence on imaging of the examination site than other detectors, according to an examination site of the image diagnostic apparatus; The medical image diagnostic system according to claim 7.
9. The processor causes the image display device to display an operation window that supports an operation related to the image by the operation detector. The medical image diagnostic system according to claim 1 .
10. an ambient light adjuster for adjusting ambient light within a bore of the imaging diagnostic device; The processor receives an operation instruction to adjust the ambient light from the operation detector and controls the ambient light adjuster. The medical image diagnostic system according to claim 1 .
11. an ambient light adjuster for adjusting ambient light within a bore of the imaging diagnostic device; The processor controls the ambient light adjuster in response to the content of the image displayed on the image display. The medical image diagnostic system according to claim 1 .
12. an ambient light adjuster for adjusting ambient light within a bore of the imaging diagnostic device; The processor controls the ambient light adjuster in synchronization with a video scene displayed on the video display. The medical image diagnostic system according to claim 1 .
13. The ambient light adjuster adjusts at least one of hue, lightness, and saturation. The medical image diagnostic system according to any one of claims 10 to 12.
14. The image display device is a projector that projects an image onto a screen visible to the subject, a head-up display, a head-mounted display, a liquid crystal display, or an organic electroluminescence display. The medical image diagnostic system according to claim 1 .
15. The imaging diagnostic device includes a magnetic resonance imaging device, an X-ray CT device, a PET device, a radiation therapy device, or a particle beam therapy device. The medical image diagnostic system according to claim 1 .
16. A method for operating a medical image diagnostic system including: an image display that displays an image in a manner that is visible to a subject during an examination of the subject by an image diagnostic device; an operation detector that detects an operation related to the image displayed on the image display; and a processor, The operation detector detects an operation related to the image by a movement of the subject during the test; receiving, by the processor, an operation instruction indicating an operation related to the image from the operation detector; a step of the processor controlling an image displayed by the image display device based on the content of the received operation instruction; A method of operating a medical imaging diagnostic system comprising:
17. The operation related to the video includes at least one of selection and switching of video content to be displayed on the video display device, fast forwarding, rewinding, pausing, playing, and adjusting the volume of the video displayed on the video display device.
17. A method of operating a medical imaging system according to claim 16.
Citation Information
Patent Citations
Magnetic resonance imaging apparatus
JP2017080299A
System and method for displaying images to a patient on a moving table during medical imaging or treatment - Patents.com
JP2020535931A
