Subject movement measuring device, subject movement measuring method, program, and medical image diagnostic device

Abstract

To provide a technique for highly accurately measuring the movement of a subject.SOLUTION: A subject movement measurement device comprises: a measurement unit which measures the movement of a subject to output movement information of the subject; an inference unit which reduces an error due to the disturbance other than the movement of the subject from the movement information of the subject obtained by the measurement unit by using a learned model; and an output unit which outputs the movement information of the subject from which the error is reduced by the inference unit. The learned model includes a function which receives input of a dataset including movement information of a plurality of degrees of freedom and outputs the movement information of the plurality of degrees of freedom from which the error is reduced.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a technique for measuring the movement of a subject. [Background technology]

[0002] A magnetic resonance imaging (MRI) device applies a radio frequency (RF) magnetic field to a subject placed in a static magnetic field, and generates images of the inside of the subject based on magnetic resonance (MR) signals generated by the subject due to the influence of the RF magnetic field.

[0003] In recent years, the resolution of images output from magnetic resonance imaging devices has been increasing. This increase in image resolution has led to artifacts that were previously relatively inconspicuous becoming more pronounced, creating a new issue for which improvement is desired. Head movement within the head coil is known to be one of the causes of such artifacts, and attempts have been made to correct the gradient magnetic field of the magnetic resonance imaging device in accordance with head movement measured by a camera (Non-Patent Document 1).

[0004] Non-Patent Document 1 discloses a method (hereinafter referred to as an optical tracking system) in which markers are attached to a moving subject, the subject is photographed from the outside with a camera, and the subject's six-degree-of-freedom movement is detected from the difference in marker positions between video frames. Non-Patent Document 2 also discloses a method in which smoothing using a Kalman filter is applied to subject movement information obtained by an optical tracking system to reduce noise. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] M. Zaitsev, C. Dold, G. Sakas, J. Hennig, and O. Speck, "Magnetic resonance imaging of freely moving objects: Prospective real-time motion correction using an external optical motion tracking system", NeuroImage 31 (2006) 1038-1050 [Non-patent document 2] J. Maclaren, O. Speck, J. Hennig, and M. Zaitsev, "A Kalman filtering framework for prospective motion correction", Proc. Intl. Soc. Mag. Reson. Med. 17 (2009) Summary of the Invention [Problem to be solved by the invention]

[0006] As in Non-Patent Document 1, when performing correction according to the movement of the head as the subject, it is necessary to accurately capture the six degrees of freedom of the movement of the head itself. However, errors can occur in the tracking measurement results due to the influence of disturbances other than the movement of the subject (e.g., camera vibration, movement of the skin or markers, etc.). This error is called "tracking error" or "tracking noise." The presence of tracking error reduces the accuracy and reliability of subsequent correction processing and control, leading to a decrease in the quality of the final image (the occurrence of artifacts).

[0007] By applying the smoothing process disclosed in Non-Patent Document 2, sudden changes in the movement of the subject can be suppressed (corrected). However, the smoothing process may correct not only the noise caused by the tracking error but also the movement of the subject.

[0008] The present invention has been made in view of the above circumstances, and has an object to provide a technique for measuring the movement of a subject with high accuracy.

[0009] Another object of the present invention is to provide a technique for reducing as much as possible errors caused by disturbances from information about the movement of a subject obtained by tracking. [Means for solving the problem]

[0010] The present disclosure provides a method for measuring the movement of a subject, Motion information of the subject including errors caused by disturbances other than the motion of the subject, the motion information including errors caused by disturbances other than the motion of the subject, and a measurement unit that outputs motion information of the subject; and a trained model that uses the trained model to calculate a pre-trained motion of the subject from the motion information of the subject obtained by the measurement unit. Typographical error Reduce the difference Output the corrected motion information An inference part that ,of The trained model is The subject Information on the movement The news is entered, The aforementioned The error is reduced Complex Number of degrees of freedom After the above correction It has the function of outputting movement information. The corrected motion information is information in which data points including the error in the motion information of the subject are corrected to data points with reduced error. The present invention includes a subject movement measuring device characterized by the above-mentioned.

[0011] The present disclosure includes a medical image diagnostic apparatus comprising the subject motion measuring device and a means for performing processing using subject motion information output from the subject motion measuring device.

[0012] The present disclosure provides a method for measuring the movement of a subject, Motion information of the subject including errors caused by disturbances other than the motion of the subject, the motion information including errors caused by disturbances other than the motion of the subject, and outputting the subject's motion information; , studies Using the trained model, the previous Typographical error Reduce the difference Output the corrected motion information Steps to ,of The trained model includes: The subject Information on the movement The news is entered, The aforementioned The error is reduced Complex Number of degrees of freedom After the above correction It has the function of outputting movement information. The corrected motion information is information in which data points including the error in the motion information of the subject are corrected to data points with reduced error. The present invention also includes a method for measuring the movement of a subject, characterized in that:

[0013] The present disclosure includes a program for causing a computer to execute each step of the subject movement measuring method. [Effects of the Invention]

[0014] According to the present invention, the movement of a subject can be measured with high accuracy. Furthermore, according to the present invention, errors caused by disturbances can be reduced as much as possible from the movement information of the subject obtained by tracking. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a magnetic resonance imaging apparatus. [Figure 2] FIG. 10 is a diagram for explaining an example of processing by a motion calculation unit. [Figure 3] FIG. 1 is a diagram showing an example of six degrees of freedom of movement of a subject. [Figure 4] FIG. 10 is a diagram for explaining a tracking error contained in the movement of a subject. [Figure 5] FIG. 1 is a diagram showing an example of the configuration of a subject movement measuring device. [Figure 6] FIG. 2 is a diagram showing an example of machine learning by a learning unit. [Figure 7] A diagram explaining supervised learning in CNN. [Figure 8] FIG. 10 is a diagram showing an example of error-containing data and training data used for learning. [Figure 9] FIG. 10 is a diagram illustrating an example of a method for generating data with errors. [Figure 10] FIG. 10 is a diagram showing an example of learning data for PMC. [Figure 11] FIG. 10 is a diagram showing another example of learning data for PMC. [Figure 12] FIG. 10 is a diagram showing an example of learning data for RMC. [Figure 13] FIG. 10 is a diagram showing an example of a result of applying a tracking error reduction process. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, an embodiment of a medical image diagnostic apparatus of the present invention will be described in detail with reference to the drawings. The medical image diagnostic apparatus may be any modality capable of imaging a subject. Specifically, the medical image diagnostic apparatus according to this embodiment may be an MRI apparatus, an X-ray computed tomography (CT) apparatus, a PET (Positron Emission Tomography) apparatus, or the like. The medical image diagnostic apparatus according to this embodiment can be applied to single modalities such as an MR / PET apparatus, a CT / PET apparatus, an MR / SPECT apparatus, and a CT / SPECT apparatus. Alternatively, the medical image diagnostic apparatus according to this embodiment can be applied to combined modalities such as an MR / PET apparatus, a CT / PET apparatus, an MR / SPECT apparatus, and a CT / SPECT apparatus.

[0017] In the following embodiment, an example will be described in which, when imaging the head of a subject using an MRI device, head movement is measured using a tracking system and the MRI imaging conditions are corrected according to the head movement. In this example, the subject's head corresponds to the "subject." Note that the subject is not limited to the head, but may be another part of the body or the entire body. Furthermore, the subject is not limited to the human body, but may be another living organism such as an animal.

[0018] <Overall configuration of medical imaging diagnostic equipment> 1 is a diagram showing the configuration of a magnetic resonance imaging apparatus 100, which is a medical image diagnostic apparatus according to an embodiment. The magnetic resonance imaging apparatus 100 includes a static magnetic field magnet 1, a gradient magnetic field coil 2, a gradient magnetic field power supply 3, a bed 4, a bed control unit 5, RF coil units 6a, 6b, and 6c, a transmitter 7, a switching circuit 8, a receiver 9, and a radio frequency pulse / gradient magnetic field control unit 10. The magnetic resonance imaging apparatus 100 further includes a computer system 11, an optical imaging unit 21, a movement calculation unit 23, and an information processing device 200.

[0019] The static magnetic field magnet 1 has a hollow cylindrical shape and generates a uniform static magnetic field in the internal space. For example, a superconducting magnet or the like is used as this static magnetic field magnet 1.

[0020] The gradient coil 2 has a hollow cylindrical shape and is disposed inside the static magnetic field magnet 1. The gradient coil 2 is a combination of three types of coils corresponding to the mutually orthogonal X, Y, and Z axes. The gradient coil 2 generates gradient magnetic fields whose field strengths are gradient along the X, Y, and Z axes when the three types of coils are individually supplied with current from the gradient magnetic field power supply 3. The Z-axis direction is, for example, parallel to the static magnetic field direction. The gradient magnetic fields of the X, Y, and Z axes correspond, for example, to a slice selection gradient magnetic field Gs, a phase encoding gradient magnetic field Ge, and a readout gradient magnetic field Gr, respectively. The slice selection gradient magnetic field Gs is used to arbitrarily determine the imaging cross section. The phase encoding gradient magnetic field Ge is used to change the phase of the magnetic resonance signal according to the spatial position. The readout gradient magnetic field Gr is used to change the frequency of the magnetic resonance signal according to the spatial position.

[0021] The subject 1000 is inserted into the space (imaging space) inside the gradient magnetic field coil 2 while lying supine on the top plate 41 of the bed 4. This imaging space is called the inside of the bore. The bed 4 moves the top plate 41 in its longitudinal direction (left and right in FIG. 1) and up and down under the control of the bed control unit 5. The bed 4 is usually placed so that the longitudinal direction is parallel to the central axis of the static magnetic field magnet 1.

[0022] The RF coil unit 6a is for transmission. The RF coil unit 6a is configured by accommodating one or more coils in a cylindrical case. The RF coil unit 6a is disposed inside the gradient magnetic field coil 2. The RF coil unit 6a receives a high frequency signal (RF signal) from the transmitter 7 and generates a high frequency magnetic field (RF magnetic field). The RF coil unit 6a can generate an RF magnetic field in a wide area that includes most of the subject 1000. In other words, the RF coil unit 6a is equipped with a so-called whole body (WB) coil. do.

[0023] The RF coil unit 6b is for receiving. The RF coil unit 6b is placed on the tabletop 41, built into the tabletop 41, or attached to the subject 1000a of the subject 1000. During imaging, it is inserted into the imaging space together with the subject 1000a. Various types of RF coil units 6b can be attached arbitrarily. The RF coil unit 6b detects magnetic resonance signals generated in the subject 1000a. One specifically for use with the head is called a head RF coil.

[0024] The RF coil unit 6c is for transmitting and receiving. The RF coil unit 6c is placed on the tabletop 41, built into the tabletop 41, or worn by the subject 1000. During imaging, it is inserted into the imaging space together with the subject 1000. Various types of RF coil units 6c can be attached. The RF coil unit 6c receives an RF signal from the transmitter 7 and generates an RF magnetic field. The RF coil unit 6c also detects magnetic resonance signals generated in the subject 1000. An array coil formed by arranging multiple coil elements can be used as the RF coil unit 6c. The RF coil unit 6c is smaller than the RF coil unit 6a and generates an RF magnetic field that only includes a local area of ​​the subject 1000. In other words, the RF coil unit 6c includes a local coil. A local transmitting and receiving coil may be used as the head coil.

[0025] The transmitter 7 selectively supplies an RF pulse corresponding to the Larmor frequency to the RF coil unit 6 a or the RF coil unit 6 c. The transmitter 7 differentiates the amplitude and phase of the RF pulse supplied to the RF coil unit 6 a and the RF pulse supplied to the RF coil unit 6 c in accordance with the difference in the magnitude of the RF magnetic field to be formed.

[0026] A switching circuit 8 connects the RF coil unit 6c to the transmitter 7 during a transmission period when an RF magnetic field should be generated, and to the receiver 9 during a reception period when a magnetic resonance signal should be detected. The transmission period and reception period are instructed by a computer system 11. The receiver 9 performs processing such as amplification, phase detection, and analog-to-digital conversion on the magnetic resonance signals detected by the RF coil units 6b and 6c to obtain magnetic resonance data.

[0027] The computer system 11 includes an interface unit 11a, a data acquisition unit 11b, a reconstruction unit 11c, a memory unit 11d, a display unit 11e, an input unit 11f, and a main control unit 11g. The interface unit 11a is connected to the radio frequency pulse / gradient magnetic field control unit 10, the bed control unit 5, the transmitter 7, the switching circuit 8, the receiver 9, and other components. The interface unit 11a inputs and outputs signals between these connected components and the computer system 11. The data acquisition unit 11b acquires magnetic resonance data output from the receiver 9. The data acquisition unit 11b stores the acquired magnetic resonance data in the memory unit 11d. The reconstruction unit 11c performs post-processing, i.e., reconstruction such as Fourier transform, on the magnetic resonance data stored in the memory unit 11d to obtain spectrum data or MR image data of desired nuclear spins in the subject 1000. The memory unit 11d stores the magnetic resonance data and spectrum data or image data for each subject. The display unit 11e displays various types of information such as spectrum data or image data under the control of the main control unit 11g. A display device such as a liquid crystal display can be used as the display unit 11e. The input unit 11f accepts various commands and information input from an operator. As the input unit 11f, a pointing device such as a mouse or a trackball, a selection device such as a mode changeover switch, or an input device such as a keyboard can be used as appropriate. The main control unit 11g has a CPU (processor), memory, etc. (not shown), and controls the magnetic resonance imaging apparatus 100 overall.

[0028] The radio frequency pulse / gradient magnetic field control unit 10 generates the required pulse signals under the control of the main control unit 11g. The gradient magnetic field power supply 3 and the transmitter 7 are controlled so as to change each gradient magnetic field and transmit RF pulses according to the sequence. Each gradient magnetic field can also be changed based on the movement information of the subject 1000a sent from the movement calculation unit 23 through the information processing device 200. The functions of the radio frequency pulse / gradient magnetic field control unit 10 may be integrated with the main control unit 11g.

[0029] The optical imaging unit 21, marker 22, motion calculation unit 23, and information processing device 200 detect the movement of the subject 1000a and transmit the detected movement to the radio frequency pulse / gradient magnetic field control unit 10. Based on the transmitted movement, the radio frequency pulse / gradient magnetic field control unit 10 can control the gradient magnetic field to maintain a substantially constant imaging plane. This allows image data that does not generate motion artifacts even when the subject moves. In other words, a motion-corrected image can be obtained. A motion correction method that changes the gradient magnetic field in real time in accordance with the measured subject movement and constantly maintains the imaging plane is generally called prospective motion correction. On the other hand, there is also a method that measures and records the subject's movement during MR imaging, but then uses the motion measurement data to perform motion correction on the MR images after MR imaging. This is called retrospective motion correction. Either motion correction method may be used as long as it reduces motion artifacts compared to conventional MR images.

[0030] Note that "movement" generally refers to six degrees of freedom for a rigid body in three-dimensional space, expressed as three degrees of freedom for rotation and three degrees of freedom for translation. This specification also uses an example of determining six degrees of freedom for movement, but any degrees of freedom may be used as long as they can express the movement of the subject.

[0031] The optical imaging unit 21 is generally a camera, but any sensing device capable of optically imaging or capturing an image of the subject may be used. In this embodiment, to assist in accurate capture of the subject's movement and position, a marker with a predetermined pattern printed thereon is affixed to the subject, and the marker is imaged by the optical imaging unit 21. However, the subject's movement may also be captured without using a marker by tracking the subject's own characteristic points, such as skin texture such as wrinkles and eyebrow patterns, or characteristic facial features such as the shape of the nose, the area around the eyes, and the forehead.

[0032] In a configuration using a camera as the optical imaging unit 21, the number of cameras may be one or two or more. For example, if a marker is used and the positional relationship of multiple feature points on a pattern within the marker is known, the movement of the marker can be calculated from an image acquired by a single camera. If a marker is not used, or if the positional relationship of feature points is not known even when a marker is used, it is preferable to measure three-dimensional information of the subject using so-called stereo imaging. There are various stereo imaging methods, such as passive stereo using two or more cameras and active stereo combining a projector and a camera, and any method may be used. Increasing the number of cameras can improve the measurement accuracy of movement along various axes.

[0033] Furthermore, it is desirable that the optical imaging unit 21 be MR-compatible. MR compatibility means that the configuration minimizes noise that affects image data during MR imaging and operates normally even in a strong magnetic field environment. For example, an electrically shielded camera that does not use magnetic materials is an example of an MR-compatible camera. Furthermore, the optical imaging unit 21 can be placed inside a bore, which is a space surrounded by a static magnetic field magnet 1 and a gradient magnetic field coil 2, as shown in FIG. 1, or can be placed outside the bore if there is no space inside the bore. The optical imaging unit 21 may be placed in any manner as long as it can capture images of the marker or subject within a predetermined imaging range (Field of View: FOV).

[0034] When a camera is used as the optical photographing unit 21, lighting (not shown) may be used. By using this, it is possible to image the marker or subject with high contrast. It is desirable that the lighting is also MR compatible, and MR compatible LED lighting or the like can be used. The lighting can be of any wavelength or wavelength band, such as white light, monochromatic light, near-infrared light, or infrared light, as long as it can image the marker or subject with high contrast. However, considering the burden on the subject, near-infrared light or infrared light, which are wavelengths invisible to the naked eye, are preferred.

[0035] The movement calculation unit 23 will now be described. The movement calculation unit 23 analyzes the images captured by the optical imaging unit 21 and calculates the movement of the feature points of the marker 22 or the movement of the feature points of the subject 1000a. The movement calculation unit 23 may be configured by a computer having a processor such as a CPU, GPU, or MPU and memories such as ROM and RAM as hardware resources, and a program executed by the processor. Alternatively, the movement calculation unit 23 may be realized by an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), another complex programmable logic device (CPLD), or a simple programmable logic device (SPLD).

[0036] The optical imaging unit 21 and the movement calculation unit 23 are collectively referred to as a tracking system 300. This tracking system 300 is an example of a measurement unit that measures the movement of the subject and outputs information about the subject's movement. Note that although an optical tracking system using a camera has been described here, any tracking system may be used as long as it can measure the subject's movement in a non-contact manner. For example, a method using a magnetic sensor or a small receiving coil (tracking coil) may also be used.

[0037] An example of the processing of the motion calculation unit 23 will be described with reference to Figures 2A and 2B. Here, as shown in Figure 2A, a case will be described in which a motion measurement marker 22 is fixed to the head of the subject 1000a. The motion calculation unit 23 acquires a video of the marker from the optical imaging unit 21, and calculates the motion of the marker from each frame image of the video. It is assumed that the internal parameter matrix A (3 x 3 matrix) of the camera has been acquired in advance by calibration.

[0038] Here we will explain an example of a specific method for calculating movement. Here, we will explain using an example of one camera and a marker using a checkerboard pattern. As shown in Figure 2B, the pixel position (u i ,v i ) is calculated. Note that i is a subscript indicating that there are multiple feature points. In the following explanation, the subscript i may be omitted. In the case of a checkerboard pattern, for example, each corner is considered to be a feature point. Furthermore, the relative positional relationship of each feature point of the pattern on the marker is known, and if the coordinate system in this case is the marker coordinate system, the three-dimensional coordinates of each feature point are (m xi ,m yi ,m zi ) where the pixel position in the camera coordinate system (u i ,v i ) and the coordinates of the corresponding feature points in the marker coordinate system (m xi ,m yi ,m zi ) can be expressed by the following relation:

[0039]

number

number

[0040] The projection matrix P represents the transformation matrix from the marker coordinate system (or world coordinate system) to the camera coordinate system for the corresponding feature points. The projection matrix P has 12 variables (6 degrees of freedom). For example, if there are six or more pixel positions (u i ,v i ) and the coordinates of the corresponding feature points in the marker coordinate system (m xi ,m yi ,m zi ) the projection matrix P can be found by using the six-point algorithm that uses the relationship. Once the projection matrix P is found, any method, including nonlinear solutions, can be used.

[0041] Let P be the projection matrix calculated from the camera image acquired at a certain time t0. t0 The projection matrix calculated from the camera image acquired at time t1 of the next frame is P t1 Then, the movement of the marker between time t0 and t1 (P camera,t0→t1 )teeth, P camera,t0→t1 =P t1 P -1 t0 This allows us to calculate the six degrees of freedom (α, β, γ, t x ,t y ,t z ) movement can be calculated.

[0042] Although one example of a calculation method has been shown here, any method can be used as long as it can calculate the movement of six degrees of freedom at each time. For example, although an example of calculation from one camera image has been shown here, the movement can also be calculated using images from two or more cameras. By using images from multiple cameras in a complementary manner, the movement of six degrees of freedom can be captured with high accuracy. Furthermore, if the relative positions between feature points on the marker are unknown, the three-dimensional coordinates of each feature point can be calculated using a three-dimensional measurement means that performs stereo photography. In such a case, the three-dimensional coordinates (m xi ,m yi ,m zi ) correspond to the three-dimensional coordinates determined by a three-dimensional measurement means that performs stereoscopic imaging. Note that if a three-dimensional measurement means that performs stereoscopic imaging is used, the relative position information between feature points does not need to be known. Therefore, for example, skin texture such as wrinkles can be used as feature points to capture the six degrees of freedom of movement of the subject 1000a. In this case, there is no need to attach markers to the subject 1000a, which reduces the burden on the subject.

[0043] Generally, the movement of the subject 1000a can be determined using the above method. However, the relative positional relationship of the feature points may change during measurement for some reason, or the measured value may be affected by disturbances other than the subject's movement, such as camera vibration or movement of the skin to which a marker is attached. The error in the measured value caused by disturbance factors other than the subject's movement is called a "tracking error." Tracking error will be explained using FIGS. 3 and 4.

[0044] Assuming the subject is a rigid body, the tracking system 300 provides measurement results for six degrees of freedom (tx, ty, tz, rx, ry, rz) at each time point, as shown in Figure 3. In Figure 3, α in Figure 2A is represented as rx, β as ry, and γ as rz. Generally, the six degrees of freedom can move independently. However, for example, the head during an MRI scan cannot move freely because the subject is lying on a bed and the head is somewhat fixed by a fixture inside the head coil. These constraints make it difficult to achieve simple movements, such as moving the head only in the x-axis or rotating it only around the z-axis. Instead, the movement involves a mixture of translation and rotation of multiple degrees of freedom. Furthermore, due to these constraints, trends or patterns often emerge in the way the movements are mixed. This characteristic or correlation observed among the subject's six degrees of freedom movements is referred to herein as "linkage." 3, for example, the six degrees of freedom (tx, ty, tz, rx, ry, rz) show similar fluctuations (although there are differences in positive and negative directions and amplitudes) during the period indicated by reference numeral 301. Furthermore, during the periods indicated by reference numerals 302 and 303, the same tendency of movement is observed, with a mixture of positive rotation around the x-axis and negative rotation around the y- and z-axes.

[0045] FIG. 4 is a schematic diagram of the measurement results (circles) of the subject's motion when a known motion (solid line) is made. Here, only tx, one of the six degrees of freedom variables shown in FIG. 3, is shown. Measurements are usually not performed continuously, so they are performed at a certain time interval (here, Δt). Furthermore, the actual motion matches within the measurement accuracy (Δx). Here, measurement accuracy refers to the fact that the measured value always varies within a certain range due to camera calibration errors and image processing errors. However, in actual measurements, unwanted errors (tracking errors) exceeding the measurement accuracy are introduced due to factors such as camera vibration (the optical imaging unit 21) and the subject's skin movement. Typically, tracking errors are greater than the measurement accuracy. Therefore, motion correction using data containing tracking errors exceeding the measurement accuracy cannot reduce artifacts, resulting in image degradation. In this embodiment, processing is performed to reduce these tracking errors.

[0046] 5 is a block diagram showing the configuration of a subject motion measuring device according to this embodiment. The subject motion measuring device is composed of a tracking system (measurement unit) 300 and an information processing device 200. The tracking system 300 measures the motion of the subject and outputs subject motion information (e.g., motion information with six degrees of freedom). The information processing device 200 infers "subject motion information with reduced tracking errors" from "subject motion information including tracking errors" output from the tracking system 300, and outputs the error-reduced motion information to the medical image diagnostic device 100.

[0047] The functional configuration of the information processing device 200 will be described with reference to Fig. 5. The information processing device 200 has, as its main functions, an acquisition unit 201, an inference unit 202, a storage unit 203, and an output unit 204. The information processing device 200 may also have a learning unit 208 as necessary (if the information processing device 200 only needs to perform inference using a trained model generated in advance, there is no need to implement the learning unit 208 in the information processing device 200).

[0048] The information processing device 200 includes, for example, a processor such as a CPU, a GPU, or an MPU, a ROM, and The motion calculation unit 23 may be configured with a computer having a memory such as RAM as a hardware resource and a program executed by the processor. In this configuration, the functional blocks 201, 202, 203, 204, and 208 shown in FIG. 5 are realized by the processor executing the program. Alternatively, all or part of the functions shown in FIG. 5 may be realized by an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), another complex programmable logic device (CPLD), or a simple programmable logic device (SPLD). The hardware resource may also be shared with the motion calculation unit 23.

[0049] Each component of the information processing device 200 may be configured as an independent device, or may be configured as a function of the tracking system 300 or a function of the medical image diagnostic device 100. Furthermore, this configuration or a part of this configuration may be realized on a cloud via a network.

[0050] The acquiring unit 201 in the information processing device 200 acquires motion information of the subject to be processed from the tracking system (measurement unit) 300. The subject's motion information is given as time-series data of measurement values ​​for each of six degrees of freedom (for example, one-second data strings for each of tx, ty, tz, rx, ry, and rz). The subject's motion information output from the tracking system 300 may contain the tracking error described above. When the acquiring unit 201 acquires the subject's motion information containing the error, it transmits the data to be processed to the inferring unit 202.

[0051] The inference unit 202 in the information processing device 200 uses the trained model 210 to perform error reduction processing to reduce errors (tracking errors) caused by disturbances other than the subject's movement from the subject's movement information acquired by the acquisition unit 201. The trained model 210 stored in the storage unit 203 is preferably one that has undergone machine learning by the learning unit 208 before being incorporated into the inference unit 202. However, the information processing device 200 may also have a built-in learning unit 208, and may perform online learning using measurement results of actual patients. The trained model 210 may be stored in the storage unit 203 in advance, or may be provided via a network. The output unit 204 outputs the inference result of the inference unit 202, i.e., the subject's movement information with reduced tracking errors, to the medical image diagnostic device 100. The subject's movement information with reduced tracking errors is used in the medical image diagnostic device 100, for example, for control and image processing to reduce motion artifacts.

[0052] <Learning> Machine learning will be described with reference to Fig. 6. The learning unit 208 is provided with training data (correct answer data) 402, which is movement information that does not include tracking errors, and error-containing data 401, which is movement information that includes tracking errors. The movement information is provided as time-series data (data string) of values ​​for each of six degrees of freedom (tx, ty, tz, rx, ry, rz), for example, as shown in Fig. 3. Here, the error-containing data 401 is generated by adding tracking error components (referred to as "error information") generated by simulating disturbances (such as vibrations of the camera or device, or movement of the skin) to the movement information that does not include tracking errors.

[0053] The learning unit 208 generates learning data by assembling (linking) data with errors 401 and corresponding teacher data (correct data not including errors) 402. The learning unit 208 stores the learning data in a memory. Then, the learning unit 208 performs supervised learning using the learning data stored in the memory, and generates motion information including tracking errors and motion information including tracking errors. The learning unit 208 performs learning using a large amount of learning data, and obtains a learned model 210 as the learning result. The learned model 210 is used in the tracking error reduction process of the inference unit 202 in the information processing device 200.

[0054] The training data 402 used for learning is movement information that does not include tracking errors. The movement of a subject may be actually measured using a tracking system under an environment or conditions where no tracking errors occur, and movement information that contains almost no tracking errors may be obtained and used as the training data 402. For example, tracking errors caused by camera vibrations can be eliminated by installing the tracking system camera away from vibration sources or by installing an anti-vibration device. Furthermore, tracking measurement results that contain almost no tracking errors caused by skin movement, etc., can be obtained by performing measurements while moving only the head while forcibly fixing the facial expression. The methods for obtaining the training data 402 by actual measurement are not limited to these, and other methods may also be used. The training data 402 may be created from actual measurement data of a large number of subjects. The training data 402 may also be generated by computer simulation instead of actual measurement data. Furthermore, the training data may be augmented by data augmentation based on actual measurement data or simulation data.

[0055] The error-containing data 401 is data in which artificially generated error information has been added to movement information that does not contain tracking errors. The error information refers to tracking errors that are mixed into the movement calculated using a camera image due to disturbances such as camera vibration or skin movement being mixed into the camera image. When adding error information, it is advisable to add an error component to the camera image itself or to the coordinate data of feature points calculated from the camera image. This is because by modifying the camera image or the coordinate data of feature points and calculating six-degree-of-freedom movement information from the modified data, it is possible to superimpose a tracking error while maintaining the linkage of the six degrees-of-freedom movements. However, a simpler method of adding tracking errors is to directly modify the numerical values ​​of the six-degree-of-freedom movement information.

[0056] A specific learning algorithm is deep learning, which uses a neural network to generate features and connection weighting coefficients for learning. For example, this is realized by a convolutional neural network (CNN) with a convolutional layer. Figure 7 shows a schematic diagram of supervised learning in CNN. In supervised learning in CNN, for example, data with errors 401 is input to the CNN to perform calculations, and the connection weighting coefficients of each layer are corrected by backpropagating the error between the CNN output (inferred data) and the training data 402. This repeats the process. This makes it possible to acquire a trained model that has the ability to output motion information with reduced tracking error when motion information containing tracking error is given as input.

[0057] Figure 8 is a schematic diagram explaining examples of error-containing data 401 and training data 402 used for learning. An example of a method for generating training data 402 to be given during learning will be explained. Here, assuming a situation in which the six-degree-of-freedom movement of the head during MRI imaging is measured by optical tracking using markers and camera images, an example of simulating head movement during MRI imaging will be shown.

[0058] As mentioned above, head movement is restricted during MRI imaging, and the six degrees of freedom of movement are linked. In the simulation, this characteristic is taken into consideration and the following conditions are set. The center point of head rotation movement is selected as the reference point at a position 90 mm from the center of the eyebrows towards the back of the head, and each time the head is moved, it is randomly selected within a range of ±20 mm from the reference point. In addition, head movement is assumed to be short-term pulse-like movement or long-term movement. In the case of MRI, It is assumed that a person is lying on a bed, and that movement at the base of the neck is restricted, making it difficult to make large horizontal movements, and that head movements are primarily rotational. Since a relaxed state is assumed as the initial state, it takes longer to return to the original position than when the movement is initiated. Human muscles move more quickly when contracting with force than when relaxing and returning to the original state.

[0059] An example of such head movement is shown by the solid line in Figure 8. Figure 8 only shows an example of translation in the x direction (tx), but in reality, all movements of the six degrees of freedom (tx, ty, tz, rx, ry, rz) are calculated as shown in Figure 3. Here, the tracking speed is set to 50 Hz, and movements are calculated every 1 / 50 seconds. Specifically, the rotational and translational movements of the head are simulated by a simulation using a 3D model of the head, and the three-dimensional coordinates (m xi ,m yi ,m zi ) every 1 / 50 seconds. Then, calculate the three-dimensional coordinate (m xi ,m yi ,m zi ) pixel position in the camera coordinate system (u i ,v i ) is calculated and converted into six-degree-of-freedom head movement data (tx, ty, tz, rx, ry, rz) using the same algorithm as the movement calculation algorithm of the movement calculation unit 23. This becomes the training data 402. Note that this matches the rotational and translational movements given to the model within the calculation error range. In the simulation, it is advisable to change the movement parameters given to the head and generate various head movement data as the training data 402.

[0060] Next, the error-containing data 401 used for learning will be explained. The error-containing data 401 is the motion information used as the training data 402 to which error information has been added. Here, as the error information, an example in which the marker includes camera vibration is generated using a simulation. First, as in the case of the training data 402, a simulation using a 3D model of the head is performed to obtain the three-dimensional coordinates (m xi , m yi , m zi ) every 1 / 50 seconds. Next, as shown in FIG. 9A, for example, the three-dimensional coordinates (m xi ,m yi ,m zi ) by the camera vibration amount (δ xi ,δ yi ,δ zi ) from the normal position. Then, the 3D coordinate data (m xi +δ xi ,m yi +δ yi ,m zi +δ zi ) pixel position in the camera coordinate system (u i ,v i ) and calculates the six-degree-of-freedom movement data (tx, ty, tz, rx, ry, rz) of the head using the same algorithm as the movement calculation algorithm of the movement calculation unit 23. This becomes the movement data of the head that includes tracking errors.

[0061] The dotted line in Figure 8 shows an example of translational movement (tx) in the x direction that includes tracking error. Comparing the solid and dotted lines in Figure 8, it can be seen that they are slightly different. In the simulation, it is advisable to change the error parameters of the 3D coordinates based on the camera vibration model and generate various data. These will be the data with errors. For example, since the way the device vibrates differs for each MRI imaging mode, it is possible to prepare multiple camera vibration models that correspond to different imaging modes and generate data with errors corresponding to the vibrations that can occur in each imaging mode. In addition, it is possible to assume changes in facial expression and variations in skin movement and to calculate the error component (δ) corresponding to these.xi ,δ yi ,δ zi ) may be added to generate data with errors.

[0062] In this embodiment, training data 402 is generated by the following procedure: first, the three-dimensional coordinates of feature points (e.g., marker patterns) are calculated based on the rotational and translational movements of the head during MRI imaging, and then, using the same algorithm as the motion calculation algorithm, six-degree-of-freedom movement data is calculated from the three-dimensional coordinate data of the feature points and the corresponding camera pixel coordinates. By adopting such a procedure, training data 402 can be generated that simulates the six-degree-of-freedom movements of a subject during actual MRI imaging and their linkages. Furthermore, for data with errors, errors are added to the three-dimensional coordinates of the feature points, and then the six-degree-of-freedom movement data is calculated in the same manner using the same algorithm as the motion calculation algorithm. In this way, by adopting the procedure of adding error components to the coordinate data of the feature points, the linkages of the six-degree-of-freedom movements can be maintained. This allows the tracking error to be superimposed while holding the sensor, making it possible to prepare highly valid training data.

[0063] In this embodiment, we have adopted a method of generating motion information containing tracking errors, i.e., data with errors, by adding errors to the three-dimensional coordinates of feature points as described above. However, any method can be used to generate data with errors as long as motion information containing tracking errors is generated. For example, as shown in FIG. 9B , tracking errors can be added directly to the six-degree-of-freedom motion data (tx, ty, tz, rx, ry, rz) itself, without considering a model such as feature points. In the example of FIG. 9B , outliers (black circles) are added to motion data tx (white circles) that does not contain tracking errors and is obtained by actual measurement or simulation. Data with errors is generated by adding tracking errors in this way.

[0064] In this example, 2000 patterns of training data for 10 seconds of head movement were generated, and data with errors was generated from each training data. These sets of data with errors and training data were used for learning.

[0065] Next, a learning method using data with errors and training data will be described. As mentioned above, there are two methods for correcting subject motion in an MRI apparatus: Prospective Motion Correction (PMC) and Retrospective Motion Correction (RMC). When applying the tracking error reduction process of this embodiment to the PMC method and when applying the same process to the RMC method, the input data available for inference processing in the inference unit 202 differs, so it is necessary to design learning data accordingly.

[0066] In the PMC method, the gradient magnetic field is changed in real time in accordance with the subject's movement measured by the tracking system. Therefore, the inference unit 202 must sequentially perform processing to reduce tracking errors on the subject's movement information output from the tracking system and output the error-reduced movement information to the radio frequency pulse / gradient magnetic field control unit 10. In this case, the inference unit 202 applies inference processing to the latest measurement data, so there is a restriction that past data can be used for inference processing, but future data cannot be used.

[0067] FIG. 10 shows an example of training data for PMC. The input data is a set of error-containing data with six degrees of freedom (tx, ty, tz, rx, ry, rz). The error-containing data for each degree of freedom is time-series data including error-containing data at the target time (open circles) and error-containing data from before the target time (open triangles). The target time refers to the measurement time of the data that is the target of the error reduction process (inference process). In this example, the past 50 data points (i.e., 1 second's worth of time-series data) were used. On the other hand, the correct answer data is the training data with six degrees of freedom at the target time (black circles). By using this training data and repeating calculations and learning for each measurement time, a trained model can be generated that estimates the subject's movement with reduced tracking error in real time from the time-series data of the subject's movement information for a predetermined period of time. Using such a trained model, inference can be performed taking into account the temporal continuity of the subject's movement and the interrelationship between multiple degrees of freedom, thereby reducing tracking error with high accuracy.

[0068] Note that the training data in FIG. 10 is merely an example. For example, instead of simultaneously training all six-degree-of-freedom movements, training may be performed for each combination of two or more degrees of freedom selected from the six degrees of freedom. In this case, a trained model is generated for each combination of degrees of freedom (for example, in the case of three combinations of tx and rx, ty and ry, and tz and rz, three trained models are generated). If a combination of degrees of freedom that exhibits strong interlocking is known in advance, it is advisable to train using that combination. FIG. 11A shows an example of training a set of two-degree-of-freedom movements, tx and rx.

[0069] 10, the past 50 points (1 second worth) of data are given as input data, but the number of points of past data is arbitrary, and past data for a period longer than 1 second may be given, or past data less than 1 second or just the previous 1 point of past data may be given. Alternatively, as shown in FIG. 11B, a configuration may be used in which no past data is used and only data with error at the target time is given as input data.

[0070] 10, past error-containing data is provided as past data, but past inferred data may be provided instead. That is, a combination of error-containing data at the target time and inferred data from before the target time is used as input data.

[0071] FIG. 12 shows an example of training data for RMC. In the RMC method, accumulated measurement data is post-processed to correct for the subject's motion. Therefore, it is possible to use not only data from the past, but also future data, relative to the data to be subjected to error reduction processing, for inference. Therefore, as shown in FIG. 12, for example, it is recommended to use input data consisting of past data with errors (open triangles), data with errors at a target time (open circles), and data with errors in the future (black triangles), i.e., time-series data for a predetermined period of time including both before and after the target time, for training. In this case, it is also desirable to simultaneously train movements of multiple degrees of freedom. This is because inference taking into account the interrelationships between multiple degrees of freedom enables more accurate reduction of tracking errors.

[0072] In RMC learning, as in the case of the PMC method, learning may be performed for each combination of two or more degrees of freedom selected from six degrees of freedom. Furthermore, as shown in FIGS. 10 and 11A, time-series data consisting of past data and data at a target time may be used for learning. Furthermore, as shown in FIG. 11B, learning may be performed using only data at a target time without using past data. Furthermore, past inference data may be provided as past data instead of past data with errors. In other words, a combination of past inference data, data with errors at a target time, and future data with errors may be used as input data.

[0073] It is desirable that the learning be performed by a parallel processing device that is configured with a large-scale parallel simultaneous receiving circuit or a large-capacity memory, a high-performance GPU (Graphics Processing Unit), etc. By storing a learned model learned by a high-performance learning device in the storage unit 203, it becomes possible to perform tracking error reduction processing even with a relatively simple device that does not have a large-scale, expensive hardware configuration.

[0074] <Inference> In the case of the PMC method, which requires real-time processing, the inference unit 202 reduces errors from the subject's motion information calculated based on newly captured camera images. The subject's motion information in the newly captured camera images may contain error components. The inference unit 202 uses the trained model to estimate motion information in which the error components have been reduced from the subject's motion information in the newly captured camera images.

[0075] The format of the data input to the inference unit 202 is the same as the format of the data used for training the trained model. For example, when training is performed using the data in FIG. 10, a data set consisting of the current six-degree-of-freedom subject movement calculated based on newly captured camera images and the six-degree-of-freedom subject movement calculated based on past camera images is input to the inference unit 202. The inference unit 202 estimates and outputs the six-degree-of-freedom subject movement with reduced error based on the six-degree-of-freedom subject movement for a predetermined period from the past to the present and the trained model. According to this method, time-series data is given as input data, so that inference can be made taking into account the temporal continuity of the subject movement, and it can be expected that, for example, when a sudden disturbance (outlier) appears in the measurement data due to an external disturbance, the data can be corrected to a reasonable value. In addition, By using a trained model with six degrees of freedom input and multiple degrees of freedom output, it is possible to make inferences that take into account the interrelationships between multiple degrees of freedom, thereby reducing tracking errors with greater accuracy. This method is effective in cases where the subject's movement is restricted, such as in MRI imaging, and correlations and trends can be seen between the subject's six degrees of freedom of movement.

[0076] 11A, in the case of a trained model trained using data with two degrees of freedom, data with two degrees of freedom is also used as input to the inference unit 202. When there are multiple trained models with different combinations of degrees of freedom, the inference unit 202 can execute inference processing using each trained model. This makes it possible to obtain six degrees of freedom of subject movement with reduced error.

[0077] 11B, if past data is not used for learning, only the current subject movement calculated based on newly captured camera images is input to the inference unit 202. If past data is not used, inference that takes into account the temporal continuity of the subject's movement is not possible, but inference that takes into account the linkage between multiple degrees of freedom is possible, so a certain level of accuracy can be expected in situations such as MRI imaging.

[0078] Furthermore, in the case of a trained model that uses a combination of past inference data and data with errors at the target time for training, the inference unit 202 recursively uses its own past estimates as input. That is, a data set consisting of current subject movement information calculated based on newly captured camera images and past subject movement information with errors reduced by the inference unit 202 is input to the inference unit 202. If data with errors is provided as past data, tracking errors contained in the past data may adversely affect the inference process at the target time. In contrast, by using inference data (i.e., data with reduced tracking errors) as past data, it is expected that errors in the data at the target time can be reduced with higher accuracy.

[0079] On the other hand, in the case of the RMC method, the inference unit 202 performs a tracking error reduction process on measurement data accumulated by MRI imaging. For example, if learning is performed using the data in Fig. 12, a set of time-series data for a predetermined period of time including before and after the target time is input to the inference unit 202. The inference unit 202 reduces or removes error components contained in the subject's movement at the target time based on the input data set and the learned model.

[0080] Figure 13 shows an example of the results of applying the tracking error reduction process. The solid line in the figure represents head movement that does not include tracking error (ground truth data), the dotted line represents head movement that includes tracking error (input data), and the dashed line represents the output result (inferred data) of the inference unit 202. Comparing the dotted and dashed lines, it is clear that the dashed line is closer to the solid line, and that the error has been reduced.

[0081] The error reduction process of this embodiment reduces errors contained in the tracking measurement results, thereby enabling highly accurate measurement of the subject's movement. If highly accurate measurement results of the subject's movement can be obtained, they can be used to control a medical image diagnostic device (for example, to correct the gradient magnetic field of an MRI device) or for image reconstruction, thereby enabling the acquisition of high-resolution images with fewer artifacts.

[0082] (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0083] 200: Information processing device 202: Reasoning part 204: Output section 210: Trained model 300: Tracking system (measurement unit) 1000a:Subject

Claims

1. a measurement unit that measures a subject's motion and outputs motion information of the subject having a plurality of degrees of freedom, the motion information including errors caused by disturbances other than the subject's motion; an inference unit that uses a trained model to output corrected motion information in which the error is reduced from the motion information of the subject obtained by the measurement unit, the trained model has a function of receiving motion information of the subject and outputting the corrected motion information of the plurality of degrees of freedom in which the error has been reduced; The corrected motion information is information in which data points including the error in the motion information of the subject are corrected to data points with the error reduced. A subject movement measuring device characterized by:

2. 2. The subject motion measuring device according to claim 1, wherein the plurality of degrees of freedom are six degrees of freedom: three translational degrees of freedom and three rotational degrees of freedom.

3. The subject movement measuring device according to claim 1 or 2, wherein the trained model is trained using a set of time-series data of the movement information for a predetermined period of time.

4. 4. The subject movement measuring device according to claim 3, wherein the time series data for the predetermined time period includes a data point including the error and data that precedes the data point including the error.

5. The time series data for the predetermined time period includes the data points including the errors, data in the past of the data points including the errors, and data in the future of the data points including the errors.

4. The subject movement measuring device according to claim 3.

6. The inference unit uses the movement information obtained by the measurement unit as data that is older than the data point including the error.

6. The subject movement measuring device according to claim 4 or 5.

7. 6. The subject movement measuring device according to claim 4, wherein the inference unit uses the corrected movement information as data older than the data point including the error.

8. A subject movement measuring device described in any one of claims 1 to 7, characterized in that the learned model is learned by supervised learning using learning data including error-containing data, which is movement information including the error, and teacher data, which is movement information not including the error.

9. 9. The subject movement measuring device according to claim 8, wherein the data with errors is data generated by adding error information generated by simulating a disturbance to the movement information not containing the errors.

10. the measurement unit includes a camera that captures an image of the subject, and a movement calculation unit that calculates three-dimensional coordinates of feature points of the subject from an image captured by the camera and converts the three-dimensional coordinates of the feature points into the movement information; 10. The subject motion measuring device according to claim 9, wherein the data with errors is data generated by adding the error information to three-dimensional coordinates of the feature points in motion that does not include errors, and then converting the three-dimensional coordinates of the feature points to which the error information has been added into motion information of a plurality of degrees of freedom using the same algorithm as that of the motion calculation unit.

11. 11. The subject movement measuring device according to claim 1, wherein the measuring unit measures the movement of a person's head as the subject movement.

12. 12. The subject movement measuring device according to claim 11, wherein the error caused by the disturbance includes an error caused by vibration of the measuring unit or an error caused by movement of the subject's skin.

13. A subject movement measuring device according to any one of claims 1 to 12; means for performing processing using subject movement information output from the subject movement measuring device; A medical image diagnostic apparatus comprising:

14. 14. The medical image diagnostic apparatus according to claim 13, wherein the medical image diagnostic apparatus is a magnetic resonance imaging apparatus.

15. A control means for controlling the gradient magnetic field generated by the magnetic field generating means, 15. The medical image diagnostic apparatus according to claim 14, wherein the control means controls the gradient magnetic field based on the corrected motion information.

16. A receiving means for receiving a magnetic resonance signal; and an image constructing means for constructing an image based on the magnetic resonance signals received by the receiving means, 15. The medical image diagnostic apparatus according to claim 14, wherein said image constructing means constructs said image based on said corrected motion information.

17. measuring a motion of a subject, and outputting motion information of the subject having a plurality of degrees of freedom, the motion information including errors caused by disturbances other than the motion of the subject; and outputting corrected motion information in which the error is reduced from the motion information of the subject using a trained model; the trained model has a function of receiving motion information of the subject and outputting the corrected motion information of the plurality of degrees of freedom in which the error has been reduced; The corrected motion information is information in which data points including the error in the motion information of the subject are corrected to data points with the error reduced. A method for measuring the movement of a subject.

18. A program for causing a computer to execute each step of the subject movement measuring method according to claim 17.

Images