Brain information display device and brain information display method

Abstract

The present invention provides a brain information display device and a brain information display method that can generate a hologram as a three-dimensional image showing brain information that forms a three-dimensional curved surface from brain image information, etc. [Solution] The brain information display device 1 comprises a blade, an LED device arranged on the blade, a rotation drive unit, and a control unit 20. The control unit 20 includes an information acquisition unit 41 that acquires brain image information, video information, and / or functional magnetic resonance imaging information obtained by functional magnetic resonance imaging; a coordinate information generation unit 42 that generates three-dimensional coordinate information of the brain information displayed in the image information etc. acquired by the information acquisition unit; a polar coordinate conversion unit 43 that converts the three-dimensional coordinate information generated by the coordinate information generation unit into polar coordinates for a blade-type hologram display; and a hologram generation unit 44 that controls the blade and LED device to generate a hologram as a three-dimensional image showing brain information.

Description

Technical Field

[0001] The present invention relates to a brain information display device and a brain information display method.

Background Art

[0002] Conventionally, as shown in Patent Document 1, a technique has been disclosed in which about 300 to 2,500 tomographic images such as CT images in the human body are acquired, and a multiplex hologram is formed based on these tomographic images so that human organs and the like can be viewed stereoscopically.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, there is a problem that acquiring about 300 to 2,500 tomographic images such as CT images may impose a large burden and risk on patients or subjects. Therefore, a technique with a lower load and simplicity is desired.

[0005] In addition, for doctors, trainees, etc. who perform brain surgery or the like, being able to confirm the three-dimensional image of the brain in advance brings merits such as reducing the risk of accidents and improving the learning efficiency. Therefore, the necessity of a technique capable of easily displaying brain information as a hologram has been regarded as a problem.

[0006] [ The present invention has been made to solve such problems, and an object thereof is to provide a brain information display device and a brain information display method capable of generating a hologram as a three-dimensional image showing brain information that forms a three-dimensional curved surface from brain image information and the like.

Means for Solving the Problems

[0007] To achieve the above objective, according to one embodiment of the present invention, a brain information display device for displaying brain information comprises a rotatably arranged blade, an LED device arranged on the blade, a rotation drive unit for rotating the blade, and a control unit for controlling the rotation drive unit and the LED device to generate a hologram as a three-dimensional image, wherein the control unit comprises an information acquisition unit for acquiring image information, video information and / or functional magnetic resonance imaging information of the brain by functional magnetic resonance imaging, a coordinate information generation unit for generating three-dimensional coordinate information of the brain information displayed in the image information, video information and / or functional magnetic resonance imaging information acquired by the information acquisition unit, a polar coordinate conversion unit for converting the three-dimensional coordinate information generated by the coordinate information generation unit into polar coordinates for a blade-type hologram display, and a hologram generation unit for controlling the blade and the LED device to generate a hologram as a three-dimensional image showing the brain information. According to the embodiment of the present invention formed in this manner, the information acquisition unit acquires acquired brain image information, video information, and / or functional magnetic resonance imaging information using functional magnetic resonance imaging, the coordinate information generation unit generates three-dimensional coordinate information of the brain information, and the polar coordinate conversion unit converts the three-dimensional coordinate information into polar coordinates for a blade-type hologram display. As a result, the hologram generation unit can generate a hologram as a three-dimensional image representing the brain information from the brain image information, etc. Thus, it is possible to generate a hologram as a three-dimensional image representing the brain information that forms a three-dimensional curved surface from the brain image information, etc. Furthermore, the hologram allows for the recognition of brain information in a form closer to the actual three-dimensional shape, which can contribute to improving surgical accuracy and learning of events for medical professionals.

[0008] According to one embodiment of the present invention, preferably a brain information display method for displaying brain information using a brain information display device, wherein the brain information display device comprises a rotatably arranged blade, an LED device arranged on the blade, a rotation drive unit for rotating the blade, and a control unit for controlling the rotation drive unit and the LED device to generate a hologram as a three-dimensional image, and the brain information display method comprises an information acquisition step for acquiring image information, video information and / or functional magnetic resonance imaging information of the brain by functional magnetic resonance imaging, a coordinate information generation step for generating three-dimensional coordinate information of the brain information displayed in the image information, video information and / or functional magnetic resonance imaging information acquired by the information acquisition unit, a polar coordinate transformation step for converting the three-dimensional coordinate information generated by the coordinate information generation unit into polar coordinates for a blade-type hologram display, and a hologram generation step for generating a hologram as a three-dimensional image showing the brain information. According to the embodiment of the present invention formed in this manner, the information acquisition step acquires acquired brain image information, video information, and / or functional magnetic resonance imaging information by functional magnetic resonance imaging, the coordinate information generation step generates three-dimensional coordinate information of the brain information, and the polar coordinate transformation step converts the three-dimensional coordinate information into polar coordinates for a blade-type hologram display. As a result, a hologram as a three-dimensional image representing brain information can be generated from brain image information, etc., in the hologram generation step. Therefore, a hologram as a three-dimensional image representing brain information forming a three-dimensional curved surface can be generated from brain image information, etc. Furthermore, the hologram allows for the recognition of brain information in a form closer to the actual three-dimensional shape, which can contribute to improving surgical accuracy and learning of events for medical professionals. [Effects of the Invention]

[0009] According to the brain information display device and brain information display method of the present invention, a hologram can be generated as a three-dimensional image showing brain information that forms a three-dimensional curved surface from brain image information, etc. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram illustrating the outline of a brain information display device according to one embodiment of the present invention. [Figure 2] This figure shows an example of displaying a hologram of brain information in a brain information display device according to one embodiment of the present invention. [Figure 3] This figure shows how a brain information display device according to one embodiment of the present invention is mounted on a wall. [Figure 4] This is a block diagram showing the configuration of the control unit of a brain information display device according to one embodiment of the present invention. [Figure 5] This figure shows a flowchart of a brain information display method related to a brain information display device according to one embodiment of the present invention. [Figure 6] This figure shows how a control unit for a brain information display device according to one embodiment of the present invention connects to a server device via the internet. [Modes for carrying out the invention]

[0011] Hereinafter, with reference to the attached drawings, a brain information display device 1 according to one embodiment of the present invention will be described. The embodiments described herein are illustrative and will be apparent to those skilled in the art that many modifications, changes, and substitutions are possible within the spirit and scope of the present invention. Therefore, the present invention is not limited to the embodiments disclosed, and various modifications, changes, etc., are possible in its form and details without departing from the claims. Furthermore, the components disclosed in the specification can be freely combined.

[0012] As shown in Figure 1, the brain information display device 1 according to one embodiment of the present invention is formed as a device that displays information of a three-dimensionally curved brain in the air as a hologram, a three-dimensional image. The brain information display device 1 forms a blade-type hologram display device, for example, a blade-type 3D hologram display device. The brain information display device 1 has a pseudo-hologram function that utilizes the rotational afterimage based on a high-speed rotating blade 2. Note that the term "hologram" is used to include pseudo-holograms. Brain information includes the general structure of the brain, the general structure of the brain shown without the human skull, and the external and internal structures of each part of the brain. It may also include brain-related information such as structures other than the brain in the human head, such as blood vessels, bones, and tumors. Brain information may also show the state of the skull and the general structure of parts of the brain during craniotomy. Other brain-related information, such as instruments used during brain surgery, can also be displayed by the brain information display device 1. This makes it easier to grasp and recognize the position of surgical instruments in three dimensions during surgery. Furthermore, brain information is formed in a rounded shape and has a three-dimensional curved surface shape.

[0013] The brain information display device 1 comprises a blade 2, a base socket 3, an LED device 4, a rotation drive unit 5 (see Figure 3), a cylindrical support unit 6, a rotation angle detection unit 7, and a control unit 20.

[0014] Blade 2 is rotatably positioned as shown by arrow F1. Blade 2 is, for example, a blade for a blade-type hologram display, such as a blade-type 3D hologram display. Blade 2 is formed in a plate shape. Blade 2 is formed in a substantially rectangular shape when viewed from above. Blade 2 extends linearly in the horizontal direction. Blade 2 is connected to the drive shaft 5a of the rotary drive unit 5 via the base socket 3. Blade 2 is rotatably positioned by the rotary drive unit 5. Blade 2 is made of a metal such as aluminum. Blade 2 may also be made of resin, carbon, etc. Blade 2 is fixed to the base socket 3 and can rotate together with the base socket 3.

[0015] The base socket 3 forms a socket for attaching and securing the blade 2. The base socket 3 is formed to secure the blade 2 to the disc-shaped base with screws. When the base socket 3 is rotated, the blade 2 attached to the base socket 3 also rotates with it. The drive shaft 5a of the rotary drive unit 5 is engaged with and fixed to the base socket 3. The base socket 3 is positioned to rotate relative to the cylindrical support unit 6.

[0016] As shown in Figure 1, the LED device 4 is arranged on the blade 2. The LED device 4 is illustrated in Figure 1, but is omitted from Figure 2. The LED device 4 comprises, for example, a plurality of LEDs 4a arranged on the blade 2. The LEDs 4a are arranged in a line on the front surface of the blade 2. The LEDs 4a are arranged at regular intervals from one end of the blade 2 to the other. However, the LEDs 4a do not have to be arranged at regular intervals; for example, they may be arranged at a higher density closer to the ends of the blade 2. The LEDs 4a are arranged in a straight line at the top of the blade 2. The LED 4a device can turn each LED 4a on and off in any color and at any timing. The LEDs 4a form light-emitting elements and are formed, for example, by an RGB LED 4a. When an RGB LED 4a is formed, it contains a red LED chip, a green LED chip, and a blue LED chip. The LEDs 4a can produce the desired color by any combination of the red LED chip, green LED chip, and blue LED chip. The LED device 4 is also composed of LEDs 4a with relatively high brightness. The LED device 4 emits light at timings controlled by the control unit 20, projecting an image onto the human eye through the rotational afterimage effect. For example, the LED device 4 emits light in synchronization with the rotation of the blade 2, forming a hologram as a three-dimensional image through the afterimage effect. The LED device 4 is connected to a power supply (not shown), which supplies power to the LED device 4. The power supply is configured to convert power supplied from an external power source to a predetermined voltage and to supply relatively high power consumption. Power is supplied from the cylindrical support part 6 to the LED device 4 on the blade 2 by a power supply structure such as a slip ring (not shown). The slip ring electrically connects the cylindrical support part 6 and the base socket 3. This allows power from the power supply to be supplied to the LED device 4 on the rotating blade 2. The power supply structure is not limited to a slip ring and may be configured by, for example, inductive power supply. The power supply is also electrically connected to the control unit 20 and configured to supply power.

[0017] The rotation drive unit 5 is configured to rotate the blade 2. The rotation drive unit 5 includes, for example, a motor. The drive shaft 5a of the rotation drive unit 5 is connected to the base socket 3 of the blade 2. The drive shaft 5a of the rotation drive unit 5 is connected to the base socket 3 of the blade 2 and supports the base socket 3. A cylindrical support portion 6 for fixing the rotation drive unit 5 is formed around the rotation drive unit 5. Note that the drive shaft 5a of the rotation drive unit 5 may be connected to the base socket 3 via a gear. In this case, the rotation of the rotation drive unit 5 is transmitted as the rotation of the base socket 3 and the blade 2 via the gear. The rotation drive unit 5 is formed to rotate the blade 2 at a predetermined rotational speed. The rotation drive unit 5 is formed by, for example, a DC brushless motor.

[0018] The rotation angle detection unit 7 is formed to detect the rotation angle of the blade 2. The rotation angle detection unit 7 is provided, for example, on the cylindrical support portion 6 and the base socket 3. The rotation angle detection unit 7 is formed by, for example, a hall sensor 50 and a magnet 59. The hall sensor 50 is provided, for example, on the cylindrical support portion 6. The magnet 59 is provided, for example, on the base socket 3 that rotates together with the blade 2. The magnet 59 is provided, for example, on the cylindrical support portion side of the base socket 3. The rotation angle detection unit 7 can detect the timing when the magnet 59 passes through the front position of the hall sensor 50, and based on this detection timing, detects the rotation angle of the blade 2 and the base socket 3. The rotation angle detection unit 7 detects, for example, that the blade 2 has reached a predetermined reference position. The rotation angle detection unit 7 outputs the detection result to the control unit 20 as a detection signal of the rotation angle. [[ID=,5]]

[0019] The cylindrical support portion 6 is formed in a cylindrical shape. The base of the cylindrical support portion 6 is fixed to the wall W. The cylindrical support portion 6 can be installed at any place such as a stand or a floor surface other than the wall W. The cylindrical support portion 6 fixes the rotation drive unit 5 inside. The top of the cylindrical support portion 6 is arranged slightly apart from the base socket 3, but may support the base socket 3 in a slidable state by a bearing or the like. The cylindrical support portion 6 forms an outer shell for protecting the inner rotation drive unit 5.

[0020] The control unit 20 is located near the rotary drive unit 5. The control unit 20 is located, for example, on a desk T that is in contact with and connected to a wall W. The control unit 20 may also be located anywhere via the Internet. The control unit 20 controls the display operation of brain information by the brain information display device 1. More specifically, the control unit 20 controls the rotation of the rotary drive unit 5 and can control the rotational operation of the base socket 3 and blade 2, their rotational speed, and the start and stop of rotation. The control unit 20 also controls the LED device 4 and can control the lighting color (luminescence color), lighting intensity (brightness), lighting timing, and lighting duration of multiple LEDs. For example, the control unit 20 can individually light up, turn off, or light up multiple LEDs in a predetermined color based on the rotation angle of the blade 2, and can perform control to form a desired image on a display surface in the air or to the human eye viewing it through the rotational afterimage effect. The control unit 20 is, for example, a computer. The brain information display device 1 may consist of a server, a smartphone, a tablet terminal, or other information processing device. As shown in Figure 6, the brain information display device 1 may be connected to a server device 9 via the Internet 8, and the server device 9 may function as a control unit to constitute the brain information display device.

[0021] The control unit 20 incorporates a CPU 21 such as a processor and a storage device 22 such as memory, and controls connected devices to execute predetermined controls based on a predetermined control program recorded in the memory. The control unit 20 is electrically connected to the LED device 4, the rotation drive unit 5, the rotation angle detection unit 7, etc. These electrical connections may be made by wireless communication or the like.

[0022] The control unit 20 may further include an input device 23 for setting the display content and an output device 24 for checking the display status, etc. The input device 23 is, for example, an input device such as a keyboard or mouse that allows the user to input instructions such as characters into the input unit. For example, the input device 23 can be used to set the start and stop of control of the control unit 20, select predetermined images or predetermined videos, change the display color, etc. The output device 24 can display the control content of the control unit 20, the operation content or commands of the input device 23, etc. The output device 24 is, for example, a monitor having a screen that outputs and displays images or videos as an output unit. For example, the user operates an application displayed on such an output device 24 of the brain information display device 1, and the functions of the brain information display device 1 are executed.

[0023] The control unit 20 controls the rotation drive unit 5 and the LED device 4. The control unit 20 controls the rotation drive unit 5 and the LED device 4 to generate a pseudo-hologram as a three-dimensional image. The control unit 20 has the function of illuminating the LED device 4 in synchronization with the rotation of the blade 2 and at a predetermined number of rotations. The control unit controls the LED device 4 based on, for example, the detection signal of the rotation angle detection unit 7. The detection signal is, for example, a signal that includes information indicating the rotation angle of the blade 2. For example, the control unit 20 switches the illumination, extinguishing, and illumination of a predetermined color of multiple LEDs according to the rotation angle of the blade 2. As a result, the control unit 20 forms a desired pseudo-hologram in the space in front of the blade 2 due to the rotation afterimage effect. Therefore, it appears to the user that a hologram is being formed.

[0024] The storage device 22 is composed of an HDD (Hard Disk Drive). The storage device 22 may also be composed of, for example, RAM (Random Access Memory), ROM (Read Only Memory), SSD (Solid State Drive), USB memory, memory card, etc. The storage device 22 stores a predetermined program and a predetermined dataset. The predetermined program is a brain information display program that displays brain information, and causes the computer to function as the information acquisition unit 41, coordinate information generation unit 42, polar coordinate transformation unit 43, hologram generation unit 44, body information acquisition unit 51, integration function unit 52, additional information processing unit 53, part information acquisition unit 54, information supplementation unit 55, switching function unit 61, time progression display function unit 62, and display adjustment function unit 63 as described above. The predetermined program may be configured as a program that causes the computer to function as each of the function units of the control unit 20. Alternatively, the predetermined program may be configured as a program that causes the computer to function as a brain information display device or brain information display method such as (1) to (10) described later. A predetermined program may be configured such that all or part of it is stored in a server device or the like, and the output results are displayed on the brain information display device 1.

[0025] The specified program may be recorded on a computer-readable recording medium. This computer-readable recording medium may be, for example, a storage device such as an HDD, RAM, ROM, or SSD, or other data-storage-capable recording medium such as a USB memory stick, SD card, or DVD. Each program, while recorded on a computer-readable recording medium, can be executed by a computer and perform its specified function.

[0026] The control unit 20 functions as a functional unit according to a predetermined control program stored in the storage device 22, etc., as described above. Note that the controls described below are examples, and functions and controls other than those described below can also be stored in the storage device 22, etc., as predetermined control programs and are executable. The control unit 20 includes an information acquisition unit 41 that acquires image information, video information, and / or functional magnetic resonance imaging information of the brain using functional magnetic resonance imaging; a coordinate information generation unit 42 that generates three-dimensional coordinate information of the brain information displayed in the image information, video information, and / or functional magnetic resonance imaging information acquired by the information acquisition unit 41; a polar coordinate conversion unit 43 that converts the three-dimensional coordinate information generated by the coordinate information generation unit 42 into polar coordinates for a blade-type hologram display; and a hologram generation unit 44 that generates a hologram of the brain information as a three-dimensional image.

[0027] The information acquisition unit 41 acquires image information, video information, and / or functional magnetic resonance imaging information from the brain using functional magnetic resonance imaging. The information acquisition unit 41 may acquire this information from, for example, an external device or an external server. The information acquisition unit 41 may also read information stored in the storage unit. Image information is, for example, an RGB image, which is an image taken by a camera or the like. Video information is, for example, a video, which can be images taken by a camera or a smartphone. Functional magnetic resonance imaging is, for example, an fMRI image, which can be formed by a collection of images with time-series information using a device capable of taking images using functional magnetic resonance imaging. The information acquisition unit 41 enables the acquisition of brain images in various easily accessible formats, such as photographic images, video, and functional magnetic resonance imaging. Therefore, a relatively wide range of users can relatively easily acquire information and utilize the present invention. Brain image information, etc., may include images taken during surgery, etc.

[0028] The coordinate information generation unit 42 generates three-dimensional coordinate information of brain information. Brain information includes brain regions, etc. The coordinate information generation unit 42 generates three-dimensional coordinate information of brain information displayed in image information, video information, and / or functional magnetic resonance imaging information acquired by the information acquisition unit 41. The coordinate information generation unit 42 can generate three-dimensional coordinate information of brain information relatively easily from two-dimensional images and videos, and three-dimensional images and videos. The coordinate information generation unit 42 may also have a function to display the generated three-dimensional coordinate information on an output device 24 such as a monitor.

[0029] The polar coordinate conversion unit 43 converts three-dimensional coordinate information into polar coordinates. These polar coordinates are polar coordinates for a blade-type hologram display. The polar coordinate conversion unit 43 converts the three-dimensional coordinate information generated by the coordinate information generation unit 42 into polar coordinates for a blade-type hologram display. By converting three-dimensional coordinate information into polar coordinate information, the polar coordinate conversion unit 43 facilitates the three-dimensional representation of points by rotating blades and facilitates the creation of pseudo-holograms.

[0030] The hologram generation unit 44 displays based on the polar coordinates converted by the polar coordinate transformation unit 43. The hologram generation unit 44 controls the light emission of the LED device 4 to perform the display. The hologram generation unit 44 displays brain information as a hologram, a three-dimensional image. Therefore, information about the three-dimensionally curved brain can be recognized in a form closer to the actual three-dimensional shape, contributing to improved surgical accuracy and learning of phenomena for medical professionals. Furthermore, the hologram generation unit 44 can display brain information as a three-dimensional image. The hologram generation unit 44 makes it easier to visualize the three-dimensional structure and internal structure of the brain. In addition, the hologram generation unit 44 allows for an intuitive understanding of brain regions.

[0031] Furthermore, the coordinate information generation unit 42 of the control unit 20 includes a body information acquisition unit 51 that adjusts the three-dimensional coordinate information of the brain information based on the patient's body information from which the image information, video information, and / or functional magnetic resonance imaging information has been acquired. The body information acquisition unit 51 can improve the accuracy of the three-dimensional coordinate information of the brain information by correcting the coordinate system based on, for example, the patient's head size, imaging conditions, posture information, and length comparison with external devices. For example, images and videos may have different scales depending on the shooting position, but the body information acquisition unit 51 can display them based on the actual size.

[0032] The coordinate information generation unit 42 includes an integration function unit 52 that generates three-dimensional coordinate information of the brain based on a plurality of image information, video information and / or functional magnetic resonance image information. The integration function unit 52 can generate a single three-dimensional coordinate information by combining a plurality of image information, video information and / or functional magnetic resonance image information. For example, by combining image information mainly showing the right half of the brain, image information mainly showing the left half of the brain, image information mainly showing the front half of the brain and image information mainly showing the back half of the brain, the accuracy of modeling the three-dimensional coordinate information of the brain in three dimensions, such as modeling the three-dimensional coordinate information of the brain that broadly includes the right, left, front, and back sides of the brain, can be improved. Furthermore, by combining images and videos, it becomes unnecessary to capture a wide area in a single image, and a three-dimensional hologram can be formed that includes a relatively wide area, such as the right and left sides of the brain.

[0033] The coordinate information generation unit 42 includes an additional information processing unit 53 that generates three-dimensional coordinate information of the additional information G (see Figure 2) based on the image information, video information, and / or functional magnetic resonance image information to which additional information G has been added. For example, based on an image to which additional information G such as markings or written information has been added, the markings or written information can be reflected in the hologram. This makes it possible to reflect markings or marks written on the image in the three-dimensional coordinate information. For example, if a circle or the like is placed on an image of the brain, this mark can be reflected in the three-dimensional coordinate information. Therefore, the user can make the hologram easier to see by using additional markings or writings.

[0034] The coordinate information generation unit 42 includes a region information acquisition unit 54 that generates three-dimensional coordinate information by distinguishing a part of the brain B from other parts of the brain C based on the image information, video information and / or functional magnetic resonance imaging information. This allows the display method for each predetermined part or region of the brain to be changed (for example, by changing the color, displaying or not displaying, etc.), making it easier for the user to see the part they want to check. It also allows the user to check the positional relationship of a specific part with other parts and check how it looks, which can be useful in simulations for medical professionals preparing for surgery, etc. The coordinate information generation unit 42 may also have a function to recognize the outline of the outer shape of a part, etc., and display the simplified (sketched) outline of the part as a line.

[0035] The coordinate information generation unit 42 includes an information supplementation unit 55 that, when it is determined that the image information, video information, and / or functional magnetic resonance imaging information acquired by the information acquisition unit 41 is information about only a part of the entire brain, supplements information about parts not appearing in the image information, video information, and / or functional magnetic resonance imaging information from a provisional model based on the patient's physical information, and supplements the image information, video information, and / or functional magnetic resonance imaging information. The provisional model is, for example, an average brain shape and internal structure acquired and set as a reference model by size. The coordinate information generation unit 42 may, for example, have a database of average brain shape models. Thus, the control unit 20 has a provisional model function that can read and prepare a provisional model (reference model) that sets the structure of brain regions and parts as data. This makes it possible to improve the display information of the brain by supplementing the missing parts of the image from provisional model information using AI, etc. Furthermore, it is possible to estimate and supplement the structure of parts that are not visible in the image, etc., or parts that cannot be distinguished in the image, etc., due to the video or lighting conditions.

[0036] The hologram generation unit 44 includes a switching function unit 61 that switches the hologram based on a plurality of image information, video information, and / or functional magnetic resonance image information. The hologram generation unit 44 includes a time progression display function unit 62 that displays a hologram of the image information, video information and / or functional magnetic resonance image information corresponding to the passage of time, based on a plurality of the image information, video information and / or functional magnetic resonance image information. The hologram generation unit 44 also includes a display adjustment function unit 63 that displays or does not display a part B of the brain, which is generated by the part information acquisition unit, in a different color from other parts.

[0037] Next, as shown in Figure 6, a series of operations of a brain information display method that displays brain information in the air as a three-dimensional image, such as a pseudo-hologram, will be described. In this brain information display method, for example, a brain information display device 1 is used to display brain information based on brain image information, video information and / or functional magnetic resonance imaging information obtained by functional magnetic resonance imaging, on a blade-type hologram display.

[0038] As shown in Figures 2 and 5, when the brain information display method is started at the start, the process proceeds to preparation step S1. In step S1, the control unit 20 performs preparation step S1, which prepares for executing the brain information display method. In preparation step S1, the brain information display device 1 is prepared. For example, the cylindrical support unit 6, the rotation drive unit 5, etc. are fixed to the wall W, and the control unit 20 etc. are placed nearby. The blade 2 of the brain information display device 1, the base socket 3, the LED device 4, the rotation drive unit 5, the cylindrical support unit 6, the rotation angle detection unit 7, and the control unit 20 are assembled. The control unit 20 is prepared to operate the brain information display method, which displays brain information in the air as a three-dimensional image using a hologram H, for example, a pseudo-hologram. The control unit 20 may set the rotation drive unit 5 to an operable state, set the rotation speed required for controlling the rotation of the blade 2, and prepare to acquire a detection signal by the rotation angle detection unit 7. The control unit 20 performs the necessary function settings for controlling the lighting of the LED device 4 and enables the execution of a lighting control function synchronized with the rotation of the blade 2 based on the detection signal. Furthermore, for example, when a user inputs a command to start the brain information display method from the input device 23, the control unit 20 receives the command to start the brain information display method and starts executing the following predetermined steps. In the information acquisition step S2, the control unit 20 may include functions to perform, for example, the selection of data corresponding to the patient or subject to be displayed, the setting of display conditions, and initialization processing for starting the display. When step S1 is completed, the control unit 20 proceeds to S2.

[0039] In step S2, the control unit 20 executes an information acquisition step S2 in which the information acquisition unit 41 acquires brain image information, video information, and / or functional magnetic resonance imaging information. The information acquisition unit 41 of the control unit 20 acquires brain image information and video information such as RGB image information, video information, CT image information, MRI image information, and functional magnetic resonance imaging information (fMRI image information). For example, the acquired image information is illustrated by reference numeral F in Figure 2. The information acquisition unit 41 may acquire brain image information, video information, and / or functional magnetic resonance imaging information from, for example, an external storage medium, a database on a network, a cloud server, or an imaging device. The information acquisition unit 41 may store the acquired information in the storage device 22 within the control unit 20. The information acquisition unit 41 does not limit the image information to be acquired to a single type, but may acquire multiple types of image information at the same time or at different times.

[0040] The control unit 20 may include, for example, functions for identifying the brain regions to be displayed from acquired image information, classifying and organizing the brain information to be displayed (such as distinguishing and organizing brain structural information, contour shape information, functional information, blood vessels, bones, etc.), and determining the display color, display magnification, and display range based on the image information. The control unit 20 may also have functions to receive, for example, from the input device 23, for identifying the brain region to be displayed based on the acquired image information, classifying and organizing the brain information to be displayed (such as distinguishing and organizing brain structural information, contour shape information, functional information, blood vessels, bones, etc.), and receiving instructions for display color, display magnification, and display range based on the image information. When step S2 is completed, the control unit 20 proceeds to S3.

[0041] In step S3, the control unit 20 executes a coordinate information generation step in which the coordinate information generation unit 42 generates three-dimensional coordinate information of brain information, such as brain region information, displayed in the image information, video information, and / or functional magnetic resonance imaging information acquired in the information acquisition step S2. The coordinate information generation unit 42 extracts brain regions (e.g., cerebrum, cerebellum, brainstem, or specific cortical regions), regions, blood vessels, bones, surgical instruments, etc., from the acquired brain image information or video information, and generates three-dimensional coordinate information corresponding to the extracted brain information, such as for each region, each region, blood vessels, bones, etc. The coordinate information generation unit 42 has a function to generate a coordinate point cloud that constitutes the three-dimensional shape of brain information, such as a coordinate point cloud having X coordinates, Y coordinates, and Z coordinates, based on, for example, voxel information, contour information, or segmentation results of the brain image. Furthermore, certain functions of the coordinate information generation unit 42, which extracts brain regions (e.g., cerebrum, cerebellum, brainstem, or specific cortical regions), body parts, blood vessels, bones, surgical instruments, etc., from acquired brain image or video information, may be included in the functions of the information acquisition unit 41. The functions of the coordinate information generation unit 42 may also be provided by an AI program. For example, the AI ​​program may extract brain regions (e.g., cerebrum, cerebellum, brainstem, or specific cortical regions), body parts, blood vessels, bones, surgical instruments, etc., from brain image or video information, and estimate three-dimensional coordinate information corresponding to the extracted brain information, such as by region, by body part, blood vessels, bones, etc.

[0042] Furthermore, the body information acquisition unit 51 of the coordinate information generation unit 42 generates three-dimensional coordinate information of the brain based on the body information of the patient, etc., from which the image information, video information, and / or functional magnetic resonance imaging information has been acquired. The body information acquisition unit 51 can improve the accuracy of the three-dimensional coordinate information of the brain by correcting the coordinate system based on, for example, the patient's head size, imaging conditions, posture information, length comparison with external devices, etc. For example, images and videos may have different scales depending on the shooting position and angle, but the body information acquisition unit 51 can correct or display them based on the actual size.

[0043] Furthermore, the integrated function unit 52 of the coordinate information generation unit 42 can generate the three-dimensional coordinate information of the brain based on multiple image information, video information, and / or functional magnetic resonance imaging information. The integrated function unit 52 can combine multiple image information, video information, and / or functional magnetic resonance imaging information to generate a single three-dimensional coordinate information. For example, by combining image information mainly showing the right half of the brain, image information mainly showing the left half of the brain, image information mainly showing the front half of the brain, and image information mainly showing the back half of the brain, the accuracy of modeling the three-dimensional coordinate information of the brain can be further improved. In addition, by combining images and videos, it becomes unnecessary to capture an entire wide area in a single image, and a three-dimensional hologram can be formed that includes a relatively wide area, for example, the right side or left side of the brain. Furthermore, the integrated function unit 52 may, for example, integrate images or videos with different functions and properties, such as RGB image information and fMRI image information, and generate composite coordinate information that associates structural information with functional information (information on changes over time).

[0044] Furthermore, the additional information processing unit 53 of the coordinate information generation unit 42 generates the three-dimensional coordinate information of the additional information G based on the image information, video information, and / or functional magnetic resonance image information to which the additional information G has been added. For example, based on an image to which additional information G such as markings or written information has been added, the markings or written information can be reflected in the hologram. This makes it possible to reflect markings or marks written on the image in the three-dimensional coordinate information. For example, if a circle is placed on an image of the brain, this mark can be reflected in the three-dimensional coordinate information. Therefore, the user can make the displayed content of the hologram easier to see by using additional markings or writings.

[0045] Furthermore, the region information acquisition unit 54 of the coordinate information generation unit 42 generates the three-dimensional coordinate information by distinguishing a part of the brain B from other parts of the brain based on the image information, video information and / or functional magnetic resonance imaging information. The region information acquisition unit 54 can, for example, generate the three-dimensional coordinate information by distinguishing a part of the brain B, such as the cerebrum, from other parts of the brain C, such as the cerebellum. The region information acquisition unit 54 distinguishes and recognizes brain regions (e.g., cerebrum, cerebellum, brainstem, or specific cortical regions), parts, blood vessels, bones, surgical instruments, etc., on the image information or video information of the brain using a predetermined program, such as an AI program. Then, the region information acquisition unit 54 generates three-dimensional coordinate information for each part. The region information acquisition unit 54 may also generate three-dimensional coordinate information based on arbitrary classifications, for example, by function or by region (e.g., anterior region, lateral region, etc.). This allows users to change the display method for specific parts or regions of the brain (for example, by changing the color, showing or hiding certain areas), making it easier for them to see the areas they want to examine. It also allows users to check the positional relationship of specific areas with other areas and how they appear, which can be useful for medical professionals in simulations to prepare for surgeries and other procedures.

[0046] Furthermore, if the information supplementation unit 55 of the coordinate information generation unit 42 determines that the image information, video information, and / or functional magnetic resonance imaging information acquired by the information acquisition unit 41 is information about a part of the brain, it supplements information about the parts not shown in the image information, video information, and / or functional magnetic resonance imaging information from a provisional model based on the patient's physical information, and supplements the image information, video information, and / or functional magnetic resonance imaging information. This allows the display information of the brain to be improved by supplementing the missing parts of the image from provisional model information using AI, etc. In addition, it is possible to estimate and supplement the structure of parts that are not visible in the image, etc., or parts that cannot be distinguished in the image, etc., due to the video or lighting conditions. When step S3 is completed, the control unit 20 proceeds to S4.

[0047] In step S4, the control unit 20 uses the polar coordinate transformation unit 43 to perform a polar coordinate transformation step S4, which transforms the three-dimensional coordinate information generated in the coordinate information generation step S3 into polar coordinates for a blade-type hologram display. The polar coordinate transformation unit 43, for example, obtains three-dimensional coordinate information (x, y, z) from the coordinate information generation unit 42. The polar coordinate transformation unit 43 sets a coordinate system based on the rotation center of the blade 2, for example, and transforms the three-dimensional coordinate information (x, y, z) into polar coordinate information. More specifically, the polar coordinate transformation unit 43 has the function of transforming the three-dimensional coordinate information (x, y, z) into polar coordinate information that includes angular component information corresponding to the rotation angle of the blade 2 and positional component information corresponding to the radial position and height position on the blade 2. When step S4 is completed, the control unit 20 proceeds to S5.

[0048] In step S5, the control unit 20 executes a hologram generation step in which the hologram generation unit 44 controls the blade 2 and the LED device 4 to generate a hologram as a three-dimensional image representing brain information. The hologram generation unit 44 of the control unit 20 controls the rotation drive unit 5 to rotate the blade 2 at a predetermined rotation speed, and, based on the detection signal output from the rotation angle detection unit 7, for example, the rotation angle signal and / or rotation synchronization signal, turns on, turns off, controls the color of the LED 4a of the LED device 4 in the polar coordinate information generated by the polar coordinate transformation unit 43. Based on the polar coordinate information obtained in the polar coordinate transformation step S4, the hologram generation unit 44 makes the LED 4a, which is positioned at a predetermined position on the blade 2, emit light of an arbitrary color and brightness at a timing synchronized with the rotation angle of the blade 2. As a result, due to the rotation afterimage effect of the emitted LED, a pseudo-hologram H as a three-dimensional image of the brain is formed in a predetermined space in front of the blade 2 (the user feels as if hologram H is formed), and can be seen by the user.

[0049] The hologram generation unit 44 can perform the function of switching holograms H based on multiple image information, video information, and / or functional magnetic resonance image information using the switching function unit 61. For example, the hologram generation unit 44 can switch and display holograms corresponding to multiple image information, etc., at any time using the switching function unit 61. For example, it is possible to switch and display holograms H related to image information of different patients' brains, or to switch hologram displays corresponding to image information, etc., that are related to the same brain but are viewed from different angles, or to switch hologram displays corresponding to image information, etc., that are related to different parts of the brain, such as the cerebrum and cerebellum, even if the angle is the same. For example, since different holograms can be switched and displayed continuously, it is possible to easily explain the differences to the user, for example, the viewer. The switching function unit 61 has the function of switching one hologram H to another hologram H at any time by operating the input device 23.

[0050] Furthermore, the hologram generation unit 44 can, using the time progression display function unit 62, perform a function to display the hologram H of the image information, video information and / or functional magnetic resonance imaging information corresponding to the passage of time, based on a plurality of the aforementioned image information, video information and / or functional magnetic resonance imaging information. The hologram generation unit 44 can, using the time progression display function unit 62, organize the plurality of image information, etc., in chronological order and sequentially display the hologram H based on the image information, etc., corresponding to the passage of time of a part of the brain. For example, the state of the brain during surgery can be captured using multiple image information, or the state of the brain can be captured as multiple images with time information using functional magnetic resonance imaging information, and the passage of time of such multiple image information, etc., can be switched and displayed sequentially in the hologram H. This makes it easier to check the time changes of brain information in the hologram H. The hologram generation unit 44 may be switched by manual operation of the input device 23, or it may be switched at regular intervals by automatic control. The time elapsed display function unit 62 may be configured to display a slider bar on the output device 24 according to the elapsed time, and to generate a hologram H according to the position of the slider bar.

[0051] Furthermore, the hologram generation unit 44 can, via the display adjustment function unit 63, perform a function to display or hide certain parts or regions of the brain generated by the part information acquisition unit 54 in a different color from other parts. For example, the display adjustment function unit 63 can also display only the frontal lobe region of the brain with a red outline, a different color from other parts (for example, shown with a blue outline). This makes it possible to display some of the brain's information with a different color or display from other parts, making it easier for the user to see and identify parts of the brain. In addition, by controlling the display adjustment function unit 63 to hide the outer parts or regions of the brain, it is also possible to display the inner parts or regions of the brain while hiding the outer parts or regions.

[0052] According to the brain information display method described above, three-dimensional coordinate information can be generated based on brain information from brain image information, video information, and / or functional magnetic resonance imaging information, converted into polar coordinates for a blade-type hologram display, and a pseudo-hologram H as a three-dimensional image can be displayed in the air due to the rotational afterimage effect. This makes it easier for users to intuitively grasp the structure or function information of the brain. In addition, information about the three-dimensionally curved brain can be recognized in a form closer to the actual three-dimensional shape, which can contribute to improving the surgical accuracy and learning of phenomena for medical professionals. When step S5 is completed, the control unit 20 proceeds to the end.

[0053] An example of one embodiment of the present invention may be provided in the following embodiments.

[0054] (1) A brain information display device for displaying information about the brain, comprising: a rotatably arranged blade; an LED device arranged on the blade; a rotation drive unit for rotating the blade; and a control unit for controlling the rotation drive unit and the LED device to generate a hologram as a three-dimensional image, wherein the control unit comprises: an information acquisition unit for acquiring image information, video information and / or functional magnetic resonance imaging information of the brain by functional magnetic resonance imaging; a coordinate information generation unit for generating three-dimensional coordinate information of the brain information displayed in the image information, video information and / or functional magnetic resonance imaging information acquired by the information acquisition unit; a polar coordinate conversion unit for converting the three-dimensional coordinate information generated by the coordinate information generation unit into polar coordinates for a blade-type hologram display; and a hologram generation unit for controlling the blade and the LED device to generate a hologram as a three-dimensional image showing the information about the brain.

[0055] (2) The brain information display device according to (1), wherein the coordinate information generation unit of the control unit includes a body information acquisition unit that adjusts the three-dimensional coordinate information of the brain information based on the patient's body information from which the image information, the video information and / or the functional magnetic resonance imaging information has been acquired.

[0056] (3) The brain information display device according to (1), wherein the coordinate information generation unit of the control unit comprises an integrated function unit that generates the three-dimensional coordinate information of the brain based on a plurality of image information, video information and / or functional magnetic resonance image information.

[0057] (4) The brain information display device according to (1), wherein the coordinate information generation unit of the control unit comprises an additional information processing unit that generates the three-dimensional coordinate information of the additional information based on the image information, video information and / or functional magnetic resonance image information to which the additional information has been added.

[0058] (5) The brain information display device according to (1), wherein the hologram generation unit includes a switching function unit that switches the hologram based on a plurality of image information, video information and / or functional magnetic resonance image information.

[0059] (6) The brain information display device according to (1), wherein the hologram generation unit includes a time progression display function unit that displays a hologram of the image information, video information and / or functional magnetic resonance image information corresponding to the passage of time, based on a plurality of the image information, video information and / or functional magnetic resonance image information.

[0060] (7) The brain information display device according to (1), wherein the coordinate information generation unit of the control unit comprises a region information acquisition unit that generates three-dimensional coordinate information by distinguishing a part of the brain from other parts of the brain based on the image information, the video information and / or the functional magnetic resonance image information.

[0061] (8) The brain information display device according to (7), wherein the hologram generation unit is equipped with a display adjustment function unit that displays or does not display a part of the brain generated by the part information acquisition unit in a different color from other parts.

[0062] (9) The brain information display device according to (1), wherein the coordinate information generation unit is determined to be part of the information of the entire brain when the image information, video information and / or functional magnetic resonance imaging information acquired by the information acquisition unit is determined to be part of the information of the entire brain, the information supplementation unit supplements information about the portion not shown in the image information, video information and / or functional magnetic resonance imaging information from a provisional model based on the patient's physical information and supplements it with the image information, video information and / or functional magnetic resonance imaging information.

[0063] (10) A method for displaying brain information using a brain information display device, wherein the brain information display device comprises a rotatably arranged blade, an LED device arranged on the blade, a rotation drive unit for rotating the blade, and a control unit for controlling the rotation drive unit and the LED device to produce a hologram as a three-dimensional image, and the brain information display method comprises an information acquisition step of acquiring image information, video information and / or functional magnetic resonance imaging information of the brain by functional magnetic resonance imaging, a coordinate information generation step of generating three-dimensional coordinate information of the brain information (parts) displayed in the image information, video information and / or functional magnetic resonance imaging information acquired by the information acquisition unit, a polar coordinate transformation step of converting the three-dimensional coordinate information generated by the coordinate information generation unit into polar coordinates for a blade-type hologram display, and a hologram generation step of displaying the brain information as a hologram as a three-dimensional image.

[0064] The embodiments for carrying out the present invention are not limited to those described above, and further variations can be applied. Various alternative embodiments and examples will be apparent to those skilled in the art based on the disclosed technology. As a variation, as shown in Figure 6, the brain information display device 1 may be configured such that all or part of the functions of the control unit 20 of the brain information display device 1 are executed on the server device 9. In this case, the control processing of the control unit 20 may be executed on the server device 9 via the internet 8. That is, the brain information display device 1 may consist of a user-side control unit 20, for example, a computer device, and a server device 9 via the internet 8. Alternatively, the control unit 20 of the brain information display device 1 may be configured on the server device 9 via the internet 8.

[0065] As another variation, the control unit 20 may include an angle conversion function in the coordinate information generation unit 42 or the hologram generation unit 44. The angle conversion function allows the angle of the generated hologram H to be changed. For example, the front of a projector mounted on a wall is horizontal; therefore, the top of the hologram H faces the user (the viewer), and the base faces the wall. To make this hologram H easier to see, the angle conversion function rotates the stereoscopic image by, for example, 90 degrees, so that the top is at the top and the base is at the bottom, relative to the user's vertical direction. This allows for the formation of a hologram H that extends vertically, even if the blade 2 is positioned along the wall surface, making the hologram H easier for the user to see. [Explanation of Symbols]

[0066] 1: Brain Information Display Device 2: Blade 4:LED device 5: Rotary drive unit 20: Control Unit 41: Information acquisition department 42: Coordinate information generation section 43: Polar Coordinate Transformation Unit 44: Hologram generation unit 51: Physical information acquisition department 52: Integrated Function Unit 53: Additional Information Processing Unit 54: Part information acquisition section 55: Information supplement section 61: Switching function unit 62: Time elapsed display function unit 63:Display adjustment function section

Claims

1. A brain information display device that displays information from the brain, A rotatably positioned blade, An LED device arranged on the blade, A rotational drive unit that rotates the blade, The system comprises a control unit that controls the rotation drive unit and the LED device to produce a hologram as a three-dimensional image, The control unit, An information acquisition unit that acquires the aforementioned brain image information, video information and / or functional magnetic resonance imaging information obtained by functional magnetic resonance imaging, A coordinate information generation unit generates three-dimensional coordinate information of the brain information displayed in the image information, video information and / or functional magnetic resonance imaging information acquired by the information acquisition unit, A polar coordinate transformation unit that converts the three-dimensional coordinate information generated by the coordinate information generation unit into polar coordinates for a blade-type hologram display, The system includes a hologram generation unit that controls the blade and the LED device to generate the hologram as a three-dimensional image representing the brain information, Brain information display device comprising: coordinate information generation unit, when it is determined that the image information, video information and / or functional magnetic resonance imaging information acquired by the information acquisition unit is information of a part of the entire brain, an information supplementation unit that supplements information about the part not appearing in the image information, video information and / or functional magnetic resonance imaging information from a provisional model based on the patient's physical information, and supplements the image information, video information and / or functional magnetic resonance imaging information.

2. The brain information display device according to claim 1, wherein the coordinate information generation unit of the control unit includes a body information acquisition unit that adjusts the three-dimensional coordinate information of the brain information based on the head size of the patient from which the image information, video information and / or functional magnetic resonance imaging information has been acquired.

3. The brain information display device according to claim 1, wherein the coordinate information generation unit of the control unit comprises an integrated function unit that generates the three-dimensional coordinate information of the brain information based on a plurality of image information, video information and / or functional magnetic resonance image information.

4. The brain information display device according to claim 1, wherein the coordinate information generation unit of the control unit comprises an additional information processing unit that generates the three-dimensional coordinate information of the additional information based on the image information, video information and / or functional magnetic resonance image information to which the additional information has been added.

5. The brain information display device according to claim 1, wherein the hologram generation unit includes a switching function unit that switches the hologram based on a plurality of image information, video information and / or functional magnetic resonance image information.

6. The brain information display device according to claim 1, wherein the hologram generation unit includes a time progression display function unit that displays a hologram of the image information, video information and / or functional magnetic resonance image information corresponding to the passage of time, based on a plurality of the image information, video information and / or functional magnetic resonance image information.

7. The brain information display device according to claim 1, wherein the coordinate information generation unit of the control unit comprises a region information acquisition unit that generates three-dimensional coordinate information by distinguishing a part of the brain from other parts of the brain based on the image information, the video information and / or the functional magnetic resonance image information.

8. The brain information display device according to claim 7, wherein the hologram generation unit includes a display adjustment function unit that displays or does not display a part of the brain generated by the part information acquisition unit in a different color from other parts.

9. A method for displaying brain information, which displays brain information using a brain information display device, The brain information display device comprises a rotatably arranged blade, An LED device arranged on the blade, A rotational drive unit that rotates the blade, The system comprises a control unit that controls the rotation drive unit and the LED device to produce a hologram as a three-dimensional image, The aforementioned brain information display method is An information acquisition step of acquiring the aforementioned brain image information, video information and / or functional magnetic resonance imaging information by functional magnetic resonance imaging, A coordinate information generation step that generates three-dimensional coordinate information of the brain information displayed in the image information, video information and / or functional magnetic resonance imaging information acquired by the information acquisition step, A polar coordinate transformation step that converts the three-dimensional coordinate information generated by the coordinate information generation step into polar coordinates for a blade-type hologram display, The system includes a hologram generation step of generating the hologram as a three-dimensional image representing the information of the brain, A brain information display method comprising the coordinate information generation step, which, when it is determined that the image information, video information and / or functional magnetic resonance imaging information acquired by the information acquisition step is information about a part of the entire brain, an information supplementation step which supplements information about the part not appearing in the image information, video information and / or functional magnetic resonance imaging information from a provisional model based on the patient's physical information, and supplements the image information, video information and / or functional magnetic resonance imaging information.

Images