Data processing device, magnetic resonance imaging device and data processing method

Abstract

To improve image quality.SOLUTION: A data processing device of the present invention includes a processing circuit. The processing circuit executes data processing using a learned model consisting of a neural network having a division processing layer for dividing first vector data of an input complex number by second vector data of a complex number including characteristics of the first vector data, a nonlinear layer disposed in a subsequent stage of the division processing layer for executing nonlinear arithmetic processing, and a multiplication processing layer disposed in a subsequent stage of the nonlinear layer for multiplying input data by the second vector data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The embodiments disclosed in this specification and the drawings relate to a data processing apparatus, a magnetic resonance imaging apparatus, and a data processing method. [Background technology]

[0002] In machine learning using neural networks, real-valued neural networks are typically used.

[0003] However, since signal processing using complex numbers is often used in medical data processing devices such as magnetic resonance imaging devices and ultrasound diagnostic devices, the use of complex neural networks is expected to lead to a variety of applications.

[0004] In medical images such as magnetic resonance images and ultrasound images, the phase information of the image, for example, the phase gradient, is often important, while the absolute value of the phase is relatively rarely significant. For example, in a magnetic resonance imaging device, a slight difference in the center frequency appears as a phase modulation of the entire image, but in many cases, the absolute value of the phase itself is relatively unimportant.

[0005] Therefore, when applying a complex neural network to medical images, such as for noise removal or region extraction, it is desirable to configure a neural network that does not ignore the phase information of the input image, while at the same time ensuring that the output results are not significantly affected by phase modulation of the entire image. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Special Publication No. 2019-525325 Summary of the Invention [Problem to be solved by the invention]

[0007] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve image quality. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0008] A data processing device according to an embodiment includes a processing circuit that processes data using a trained model formed of a neural network having a division processing layer that divides input first vector data of complex numbers by second vector data of complex numbers containing features of the first vector data, a nonlinear layer that is located downstream of the division processing layer and performs nonlinear arithmetic processing, and a multiplication processing layer that is located downstream of the nonlinear layer and multiplies input data by the second vector data. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a data processing device according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a magnetic resonance imaging apparatus according to an embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of an ultrasonic diagnostic apparatus according to an embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a neural network according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating a neural network according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of the configuration of a data processing device according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of the configuration of a data processing device according to the second embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of the configuration of a data processing device according to the third embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of the configuration of a data processing device according to the fourth embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of the configuration of a data processing device according to the fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] (First embodiment) Hereinafter, a data processing apparatus, a magnetic resonance imaging apparatus, and a data processing method according to embodiments will be described in detail with reference to the drawings.

[0011] The configuration of a data processing device 100 according to the embodiment will be described with reference to FIG.

[0012] The data processing device 100 is a device that generates data using machine learning. As an example, the data processing device 100 is connected to various medical image diagnostic devices, such as the magnetic resonance imaging device shown in FIG. 2 and the ultrasound diagnostic device shown in FIG. 3, and performs processing of signals received from the medical image diagnostic devices, generation of trained models, execution of the trained models, etc. Note that examples of the medical image diagnostic devices to which the data processing device 100 is connected are not limited to magnetic resonance imaging devices and ultrasound diagnostic devices, and may be other devices, such as X-ray CT devices and PET devices.

[0013] The data processing device 100 is typically a medical data processing device that processes medical data. However, the embodiment is not limited to the case where the data processing device 100 is a medical data processing device. As an example, the data processing device 100 may be a device that processes magnetic resonance data that is not medical data.

[0014] The data processing device 100 includes a processing circuit 110, a memory 132, an input device 134, and a display 135. The processing circuit 110 includes a training data generation function 110a, a learning function 110b, an interface function 110c, a control function 110d, an application function 110e, an acquisition function 110f, and a scale data acquisition function 110g.

[0015] In the embodiment, the processing functions and trained models (e.g., neural networks) performed by the training data creation function 110a, learning function 110b, interface function 110c, control function 110d, application function 110e, acquisition function 110f, and scale data acquisition function 110g are stored in memory 132 in the form of computer-executable programs. The processing circuit 110 is a processor that realizes the functions corresponding to each program by reading and executing the programs from memory 132. In other words, the processing circuit 110 having read each program has the functions shown in the processing circuit 110 of FIG. 1. Furthermore, the processing circuit 110 having read a program corresponding to a trained model (neural network) can perform processing in accordance with the trained model. Note that, although FIG. 1 illustrates the functions of the processing circuit 110 being realized by a single processing circuit, the processing circuit 110 may be configured by combining multiple independent processors, and each processor may execute a program to realize the function. In other words, each of the above functions may be configured as a program, and a single processing circuit may execute each program. Furthermore, a single processing circuit may implement two or more of the functions of the processing circuit 110. As another example, a specific function may be implemented in a dedicated, independent program execution circuit.

[0016] In FIG. 1, the processing circuit 110, the training data creation function 110a, the learning function 110b, the interface function 110c, the control function 110d, the application function 110e, the acquisition function 110f, and the scale data acquisition function 110g are examples of a processing unit, a creation unit, an input unit (learning unit), a reception unit, a control unit, an application unit, an acquisition unit, and a scale data acquisition unit, respectively.

[0017] The term "processor" used in the above description refers to circuits such as a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). The processor realizes its functions by reading and executing programs stored in memory 132.

[0018] Furthermore, instead of storing a program in memory 132, the program may be directly embedded in the circuitry of the processor. In this case, the processor realizes its function by reading and executing the program embedded in the circuitry. Therefore, for example, instead of storing a trained model in memory 132, a program related to the trained model may be directly embedded in the circuitry of the processor.

[0019] In addition, when the processing circuitry 110 is incorporated into various medical image diagnostic devices, or when it performs processing in cooperation with various medical image diagnostic devices, it may have the function of simultaneously executing processing related to these devices.

[0020] The processing circuit 110 uses the training data generation function 110a to generate training data for learning based on the data and images acquired by the interface function 110c.

[0021] The processing circuit 110 uses the learning function 110b to perform learning using the training data generated by the training data creation function 110a to generate a trained model. The processing circuit 110 uses the interface function 110c to obtain data and images for image generation by the application function 110e from the memory 132.

[0022] The processing circuitry 110 uses a control function 110d to control the overall processing of the data processing device 100. Specifically, the processing circuitry 110 uses the control function 110d to control the processing of the processing circuitry 110 based on various setting requests input by an operator via the input device 134, and various control programs and various data read from the memory 132.

[0023] Furthermore, the processing circuit 110 generates an image based on the results of processing performed using the training data generation function 110a and the learning function 110b using the application function 110e. The processing circuit 110 also applies the learned model generated by the learning function 110b to the input image using the application function 110e, and generates an image based on the application result of the learned model. The processing circuit 110 also acquires various data using the acquisition function 110f. The processing circuit 110 also acquires scale data, which will be described later, using the scale data acquisition function 110g.

[0024] The memory 132 is composed of a semiconductor memory element such as a RAM (Random Access Memory), a flash memory, a hard disk, an optical disk, etc. The memory 132 is a memory for storing data such as image data for display and image data for training generated by the processing circuitry 110. The memory 132 stores various data such as control programs for performing image processing and display processing as necessary.

[0025] The input device 134 receives various instructions and information input from an operator. The input device 134 is, for example, a pointing device such as a mouse or a trackball, a selection device such as a mode change switch, or an input device such as a keyboard.

[0026] The display 135, under the control of the control function 110d, etc., displays a GUI (Graphical User Interface) for receiving input of imaging conditions, images generated by the control function 110d, etc. The display 135 is, for example, a display device such as a liquid crystal display. The display 135 is an example of a display unit. The display 135 has a mouse, a keyboard, buttons, a panel switch, a touch command screen, a foot switch, a trackball, a joystick, etc.

[0027] FIG. 2 shows an example of a magnetic resonance imaging apparatus 200 incorporating the data processing apparatus 100 according to the embodiment.

[0028] 2, the magnetic resonance imaging apparatus 200 includes a static magnetic field magnet 201, a static magnetic field power supply (not shown), a gradient magnetic field coil 203, a gradient magnetic field power supply 204, a bed 205, a bed control circuit 206, a transmission coil 207, a transmission circuit 208, a reception coil 209, a reception circuit 210, a sequence control circuit 220 (sequence control unit), and the data processing device 100 described in FIG. 1. Note that the magnetic resonance imaging apparatus 200 does not include a subject P (e.g., a human body). Also, the configuration shown in FIG. 2 is merely an example.

[0029] The static magnetic field magnet 201 is a magnet formed in a hollow, approximately cylindrical shape, and generates a static magnetic field in the internal space. The static magnetic field magnet 201 is, for example, a superconducting magnet, and is excited by receiving a current from a static magnetic field power supply. The static magnetic field power supply supplies a current to the static magnetic field magnet 201. As another example, the static magnetic field magnet 201 may be a permanent magnet, in which case the magnetic resonance imaging apparatus 200 may not be provided with a static magnetic field power supply. Furthermore, the static magnetic field power supply may be provided separately from the magnetic resonance imaging apparatus 200.

[0030] The gradient magnetic field coil 203 is a hollow, approximately cylindrical coil and is disposed inside the static magnetic field magnet 201. The gradient magnetic field coil 203 is formed by combining three coils corresponding to the mutually orthogonal X, Y, and Z axes, and these three coils are individually supplied with current from a gradient magnetic field power supply 204 to generate gradient magnetic fields whose magnetic field strengths change along the X, Y, and Z axes. The gradient magnetic fields of the X, Y, and Z axes generated by the gradient magnetic field coil 203 are, for example, a slicing gradient magnetic field Gs, a phase encoding gradient magnetic field Ge, and a readout gradient magnetic field Gr. The gradient magnetic field power supply 204 supplies current to the gradient magnetic field coil 203.

[0031] The bed 205 includes a top plate 205a on which the subject P is placed, and under the control of a bed control circuit 206, the top plate 205a is inserted into the cavity (imaging port) of the gradient magnetic field coil 203 with the subject P placed thereon. The bed 205 is usually installed so that its longitudinal direction is parallel to the central axis of the static magnetic field magnet 201. Under the control of the data processing device 100, the bed control circuit 206 drives the bed 205 to move the top plate 205a in the longitudinal direction and the up-down direction.

[0032] The transmitting coil 207 is disposed inside the gradient magnetic field coil 203, and generates a high-frequency magnetic field upon receiving RF pulses from a transmitting circuit 208. The transmitting circuit 208 supplies the transmitting coil 207 with RF pulses corresponding to a Larmor frequency determined by the type of atom of interest and the magnetic field strength.

[0033] The receiving coil 209 is disposed inside the gradient magnetic field coil 203, and receives magnetic resonance signals (hereinafter referred to as "MR signals" as necessary) emitted from the subject P due to the influence of the high frequency magnetic field. Upon receiving the magnetic resonance signals, the receiving coil 209 outputs the received magnetic resonance signals to the receiving circuit 210.

[0034] The above-described transmitting coil 207 and receiving coil 209 are merely examples. They may be configured by combining one or more of a coil having only a transmitting function, a coil having only a receiving function, or a coil having a transmitting and receiving function.

[0035] The receiving circuit 210 detects the magnetic resonance signal output from the receiving coil 209 and generates magnetic resonance data based on the detected magnetic resonance signal. Specifically, the receiving circuit 210 generates the magnetic resonance data by digitally converting the magnetic resonance signal output from the receiving coil 209. The receiving circuit 210 also transmits the generated magnetic resonance data to the sequence control circuit 220. The receiving circuit 210 may be provided on the side of a gantry device that includes the static magnetic field magnet 201, the gradient magnetic field coil 203, etc.

[0036] The sequence control circuit 220 performs imaging of the subject P by driving the gradient magnetic field power supply 204, the transmission circuit 208, and the reception circuit 210 based on the sequence information. Here, the sequence information is information that defines a procedure for performing imaging. The sequence information defines the strength of the current that the gradient magnetic field power supply 204 supplies to the gradient magnetic field coil 203 and the timing of supplying the current, the strength of the RF pulse that the transmission circuit 208 supplies to the transmission coil 207 and the timing of applying the RF pulse, and the timing of detecting a magnetic resonance signal by the reception circuit 210. For example, the sequence control circuit 220 is an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), or an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The sequence control circuit 220 is an example of a scan unit.

[0037] Furthermore, when the sequence control circuit 220 receives magnetic resonance data from the receiving circuit 210 as a result of driving the gradient magnetic field power supply 204, the transmitting circuit 208, and the receiving circuit 210 to image the subject P, the sequence control circuit 220 transfers the received magnetic resonance data to the data processing device 100. The data processing device 100 performs the overall control of the magnetic resonance imaging apparatus 200 in addition to the processing described in FIG.

[0038] 1, to explain processing performed by the data processing device 100 other than the processing described in Fig. 1, the processing circuitry 110 transmits sequence information to the sequence control circuit 220 via the interface function 110c and receives magnetic resonance data from the sequence control circuit 220. Furthermore, upon receiving the magnetic resonance data, the processing circuitry 110 having the interface function 110c stores the received magnetic resonance data in the memory 132.

[0039] The magnetic resonance data stored in the memory 132 is arranged in k-space by the control function 110d, so that the memory 132 stores the k-space data.

[0040] The memory 132 stores magnetic resonance data received by the processing circuitry 110 having the interface function 110c, k-space data arranged in k-space by the processing circuitry 110 having the control function 110d, image data generated by the processing circuitry 110 having the generation function (or application function 110e), etc.

[0041] The processing circuitry 110 performs overall control of the magnetic resonance imaging apparatus 200 using the control function 110d, and controls imaging, image generation, image display, etc. For example, the processing circuitry 110 having the control function 110d accepts input of imaging conditions (imaging parameters, etc.) on a GUI and generates sequence information according to the accepted imaging conditions. In addition, the processing circuitry 110 having the control function 110d transmits the generated sequence information to the sequence control circuit 220.

[0042] The processing circuitry 110 reads k-space data from the memory 132 using a generation function (or application function 110e) not shown in FIG. 1, and generates a magnetic resonance image by performing reconstruction processing such as a Fourier transform on the read k-space data.

[0043] 3 shows an example of the configuration of an ultrasound diagnostic device main body 300 incorporating a data processing device 100 according to an embodiment. The ultrasound diagnostic device according to an embodiment includes an ultrasound probe 305 and the ultrasound diagnostic device main body 300. The ultrasound diagnostic device main body 300 includes a transmission circuit 309, a reception circuit 311, and the above-described data processing device 100.

[0044] The ultrasonic probe 305 has a plurality of piezoelectric vibrators, which generate ultrasonic waves based on drive signals supplied from a transmission circuit 309 included in the ultrasonic diagnostic device main body 300 (described later). The plurality of piezoelectric vibrators included in the ultrasonic probe 305 also receive reflected waves from the subject P and convert them into electrical signals (reflected wave signals). The ultrasonic probe 305 also has a matching layer provided on the piezoelectric vibrators, a backing material that prevents ultrasonic waves from propagating backward from the piezoelectric vibrators, and the like. The ultrasonic probe 305 is detachably connected to the ultrasonic diagnostic device main body 300. The ultrasonic probe 305 is an example of a scanning unit.

[0045] When ultrasonic waves are transmitted from the ultrasonic probe 305 to the subject P, the transmitted ultrasonic waves are reflected successively by discontinuous surfaces of acoustic impedance in the tissues of the subject P, and are received as reflected waves by the multiple piezoelectric transducers of the ultrasonic probe 305 and converted into reflected wave signals. The amplitude of the reflected wave signal depends on the difference in acoustic impedance at the discontinuous surfaces from which the ultrasonic waves are reflected. When the transmitted ultrasonic pulse is reflected by the surface of a moving blood flow or heart wall, the reflected wave signal undergoes a frequency shift due to the Doppler effect, depending on the velocity component of the moving object relative to the direction of ultrasonic transmission.

[0046] The ultrasound diagnostic device main body 300 is a device that generates ultrasound image data based on reflected wave signals received from the ultrasound probe 305. The ultrasound diagnostic device main body 300 is a device that can generate two-dimensional ultrasound image data based on two-dimensional reflected wave signals, and can generate three-dimensional ultrasound image data based on three-dimensional reflected wave signals. However, the embodiment is also applicable to cases where the ultrasound diagnostic device 10 is a device dedicated to two-dimensional data.

[0047] As shown in FIG. 3, the ultrasound diagnostic apparatus 10 includes a transmission circuit 309, a reception circuit 311, and a data processing device 100.

[0048] The transmission circuit 309 and the reception circuit 311 control the transmission and reception of ultrasound waves performed by the ultrasonic probe 305 based on instructions from the data processing device 100, which has a control function. The transmission circuit 309 has a pulse generator, a transmission delay unit, a pulser, etc., and supplies a drive signal to the ultrasonic probe 305. The pulse generator repeatedly generates rate pulses for forming transmission ultrasound waves at a predetermined pulse repetition frequency (PRF). The transmission delay unit focuses the ultrasound waves generated from the ultrasonic probe 305 into a beam shape and provides a delay time for each piezoelectric vibrator required to determine the transmission directivity to each rate pulse generated by the pulse generator. The pulser also applies a drive signal (drive pulse) to the ultrasonic probe 305 at a timing based on the rate pulse.

[0049] That is, the transmission delay unit arbitrarily adjusts the transmission direction of the ultrasonic waves transmitted from the piezoelectric transducer surface by changing the delay time given to each rate pulse. Also, the transmission delay unit controls the position of the focal point (transmission focus) in the depth direction of the ultrasonic transmission by changing the delay time given to each rate pulse.

[0050] The receiving circuit 311 also has an amplifier circuit, an A / D (Analog / Digital) converter, a reception delay circuit, an adder, a quadrature detection circuit, etc., and performs various processes on the reflected wave signal received from the ultrasound probe 305 to generate a received signal (reflected wave data). The amplifier circuit amplifies the reflected wave signal for each channel and performs gain correction processing. The A / D converter A / D converts the gain-corrected reflected wave signal. The reception delay circuit provides the digital data with a reception delay time required to determine the reception directivity. The adder performs addition processing on the reflected wave signal to which the reception delay time has been applied by the reception delay circuit. The addition processing by the adder emphasizes the reflected wave signal from a direction corresponding to the reception directivity. The quadrature detection circuit then converts the output signal of the adder into a baseband in-phase signal (I signal, I: In-phase) and a quadrature signal (Q signal, Q: Quadrature-phase). The quadrature detection circuit then transmits the I signal and Q signal (hereinafter referred to as IQ signal) as a received signal (reflected wave data) to the processing circuit 110. Note that the quadrature detection circuit may convert the output signal of the adder into an RF (Radio Frequency) signal before transmitting it to the processing circuit 110. The IQ signal and RF signal are received signals having phase information.

[0051] When scanning a two-dimensional region within the subject P, the transmission circuitry 309 causes the ultrasonic probe 305 to transmit ultrasonic beams for scanning the two-dimensional region. Then, the reception circuitry 311 generates two-dimensional reception signals from the two-dimensional reflected wave signals received from the ultrasonic probe 305. When scanning a three-dimensional region within the subject P, the transmission circuitry 309 causes the ultrasonic probe 305 to transmit ultrasonic beams for scanning the three-dimensional region. Then, the reception circuitry 311 generates three-dimensional reception signals from the three-dimensional reflected wave signals received from the ultrasonic probe 305. The reception circuitry 311 generates reception signals based on the reflected wave signals and transmits the generated reception signals to the processing circuitry 110.

[0052] The transmission circuit 309 causes the ultrasonic probe 305 to transmit an ultrasonic beam from a predetermined transmission position (transmission scanning line). The reception circuit 311 receives, from the ultrasonic probe 305, a signal due to a reflected wave of the ultrasonic beam transmitted by the transmission circuit 309 at a predetermined reception position (reception scanning line). When parallel simultaneous reception is not performed, the transmission scanning line and the reception scanning line are the same scanning line. On the other hand, when parallel simultaneous reception is performed, when the transmission circuit 309 causes the ultrasonic probe 305 to transmit one ultrasonic beam at one transmission scanning line, the reception circuit 311 receives, as multiple reception beams, signals due to the reflected waves originating from the ultrasonic beam transmitted by the transmission circuit 309 to the ultrasonic probe 305 at multiple predetermined reception positions (reception scanning lines) simultaneously through the ultrasonic probe 305.

[0053] The data processing device 100 is connected to a transmission circuit 309 and a reception circuit 311, and in addition to the functions already shown in Fig. 1, processes signals received from the reception circuit 311, controls the transmission circuit 309, generates trained models, executes the trained models, and performs various image processing. In addition to the functions already shown in Fig. 1, the processing circuit 110 has a B-mode processing function, a Doppler processing function, a generation function, etc.

[0054] Next, the configuration of the neural network according to the embodiment will be described with reference to FIGS.

[0055] FIG. 4 shows the configuration of a neural network 7 included in a trained model according to an embodiment. As shown in FIG. 4, the neural network 7 according to an embodiment includes an input layer 7a, a linear layer 7b, a nonlinear processing layer 7c, an output layer 7d, a division processing layer 7x, and a division processing layer 7y. The neural network 7 according to an embodiment is configured by appropriately combining a plurality of these layers. Data input to the input layer 7a is typically vector data, or in other words, input vectors. These input vectors are input to intermediate layers of the neural network 7. Data output to the output layer 7d is typically vector data, or in other words, output vectors. These output vectors are output from the intermediate layers of the neural network 7. Compared to a general neural network 7a described later in FIG. 5, the neural network 7 according to an embodiment includes a division processing layer 7x and a multiplication processing layer 7y; these points will be described in detail later.

[0056] Next, a typical configuration of a neural network will be described using Figure 5. Figure 5 shows an example of interconnections between layers in a neural network 7a that is generally used in machine learning. The neural network 7a typically comprises an input layer, an output layer, and intermediate layers 3, 4, 5, etc. between the input layer and the output layer.

[0057] Here, each hidden layer is composed of a linear layer and a layer related to processing using nonlinear processing (hereinafter referred to as an activation layer or nonlinear layer). For example, hidden layer 3 is composed of a linear layer 3b and a nonlinear layer 3c, hidden layer 4 is composed of a linear layer 4b and a nonlinear layer 4c, and hidden layer 5 is composed of a linear layer 5b and a nonlinear layer 5c. Furthermore, each layer is composed of multiple nodes (neurons).

[0058] Here, the data processing device 100 according to the embodiment applies a linear layer with complex coefficients and a nonlinear activation (activation function) to medical data having complex values.

[0059] That is, each layer of the processing circuit 110 has a linear layer that performs linear operations on the input vectors input to that layer, and a nonlinear operation layer that performs nonlinear operation on the output results of the linear layer. For example, considering the hidden layer 5, which is the n-th layer, the hidden layer 5 has a linear layer 5b and a nonlinear layer 5c.

[0060] Here, the linear layer performs linear operations on the input data input to each layer. For example, the linear layer 5b performs linear operations on the input vector input to the hidden layer 5, which is the nth layer, by multiplying the number of elements of the input vector by m. n and the i-th element is the complex number z n,i Then, j is a natural number, and the complex number Σ i=1 m_n ω n,i,j z n,i +b n,j Outputs a vector whose j-th element is ω n,i,j is the weight coefficient in the n-th linear layer, b n,j is a predetermined constant known as the bias term.

[0061] Next, the nonlinear layer applies a nonlinear function to the output result of the linear layer. For example, the nonlinear layer 5c applies an activation function f to the output result of the linear layer 5b, and inputs the output result to the next layer, the n+1th layer. For example, the output result z n+1,j is the activation function f n,j Using this, it can be expressed as the following equation (1).

[0062]

number

[0063] In this way, in each layer, the neural network 7 applies a linear layer with complex coefficients and a nonlinear activation (activation function) to medical data having complex values, and outputs the results sequentially to the next layer, thereby obtaining the final output result.

[0064] 5, the processing circuitry 110 generates a trained model by using the learning function 110b to train the neural network 7, which applies a linear layer with complex coefficients and a nonlinear function to medical data having complex values, in a manner described below. The processing circuitry 110 stores the generated trained model in, for example, the memory 132 as needed.

[0065] The input vector is typically a medical image or medical image data acquired from a medical image diagnostic device. If the medical image diagnostic device is a magnetic resonance imaging device 200, the input vector is, for example, a magnetic resonance image. If the medical image diagnostic device is, for example, an ultrasound diagnostic device main body 300, the input vector is, for example, an ultrasound image.

[0066] The input vector may be a medical image, or may be various image data, projection data, intermediate data, or raw data at a stage before the medical image is generated.

[0067] The output vector may be, for example, a medical image or medical image data, or, like the input vector, may be various projection data, intermediate data, or raw data prior to the generation of a medical image. If the purpose of the neural network 7 is denoising, the data output from the output layer 2 may be, for example, an image with noise removed and higher image quality than the input image. The processing circuit 110 outputs complex vector data as the output vector.

[0068] Note that, when the neural network 7 according to the embodiment is, for example, a convolutional neural network (CNN), the input vector is data represented by a two-dimensional array of, for example, 32×32, and the output vector is data represented by a two-dimensional array of, for example, 32×32. The size of the input vector and the size of the output vector may be the same or different. Similarly, the size of the output vector in an intermediate layer may be the same as or different from the size of the input vector in that layer.

[0069] Next, generation of a trained model according to the embodiment, i.e., the learning step, will be described. The processing circuit 110 generates a trained model by, for example, performing machine learning on the neural network 7 using the learning function 110b. Here, performing machine learning means, for example, determining weights in the neural network 7 consisting of an input layer, intermediate layers 3, 4, and 5, and an output layer. Specifically, it means determining a set of coefficients characterizing the connections between the input layer and intermediate layer 3, a set of coefficients characterizing the connections between intermediate layer 3 and intermediate layer 4, ..., a set of coefficients characterizing the connections between intermediate layer 5 and the output layer. The processing circuit 110 determines these sets of coefficients using the learning function 110b, for example, by backpropagation.

[0070] The processing circuit 110 uses the learning function 110b to perform machine learning based on training data, which is teacher data consisting of input vectors and output vectors, determine weighting between each layer, and generate a trained model with the weighting determined.

[0071] In deep learning, autoencoders can be used, and in this case, the data required for machine learning does not need to be supervised data.

[0072] Next, the process of applying the trained model according to the embodiment will be described. First, the processing circuitry 110 inputs an input medical image, for example, a clinical image, into the trained model using the application function 110e. For example, the processing circuitry 110 inputs the input medical image, for example, a clinical image, into the input layer of the neural network 7, which is the trained model, using the application function 110e. Next, the processing circuitry 110 acquires data output from the output layer of the neural network 7, which is the trained model, using the application function 110e as an output medical image. The output medical image is a medical image that has undergone predetermined processing, such as noise removal. In this way, the processing circuitry 110 generates an output medical image that has undergone predetermined processing, such as noise removal, using the application function 150e. If necessary, the processing circuitry 110 may display the obtained output medical image on the display 135 using the control function 110d.

[0073] Next, the background of the embodiment will be described.

[0074] In machine learning using neural networks, real-valued neural networks are typically used. However, in medical data processing devices such as magnetic resonance imaging devices and ultrasound diagnostic devices, signal processing using complex numbers is often used, so the use of complex-valued neural networks is expected to open up a variety of applications.

[0075] In medical images such as magnetic resonance images and ultrasound images, the phase information of the image, for example, the phase gradient, is often important, while the absolute value of the phase is relatively rarely significant. For example, in a magnetic resonance imaging device, a slight difference in the center frequency appears as a phase modulation of the entire image, but in many cases, the absolute value of the phase itself is relatively unimportant.

[0076] Therefore, when applying a complex neural network to medical images, such as for noise removal or region extraction, it is desirable to configure a neural network that does not ignore the phase information of the input image, while at the same time ensuring that the output results are not significantly affected by phase modulation of the entire image.

[0077] In view of this background, the data processing device 100 according to the embodiment includes a division processing layer in the neural network 7 that performs a division process by a plurality of vector data including the features of the learning input data, and a division processing layer that performs a multiplication process by the plurality of vector data, and performs complex number scaling before and after activation in the neural network 7. This can improve learning stability against changes in phase and changes in the magnification of signal values.

[0078] Such a configuration will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the configuration of a neural network related to a trained model in a data processing device 100 according to an embodiment. The neural network related to the trained model according to an embodiment uses first vector data of complex numbers as input data during training. The first vector data of complex numbers is, for example, a magnetic resonance imaging image of complex values ​​obtained from data collected by a magnetic resonance imaging apparatus.

[0079] 6, the hidden layer 5, which is the nth layer of the neural network 7 according to the embodiment, performs linear arithmetic on an input vector 5a that receives the output result from the (n-1)th layer, and is composed of a linear layer 5b arranged before a division processing layer 5x, a division processing layer 5x arranged after the linear layer 5b, a nonlinear layer 5c arranged after the division processing layer, and a multiplication processing layer 5y arranged after the nonlinear layer 5c. The output result from the output unit of the (n-1)th division processing layer becomes the input vector 5a, and the output result from the output unit of the multiplication processing layer 5y becomes the input vector 6a of the (n+1)th layer.

[0080] The first layer receives first vector data of complex numbers, which is input data during training, and the final output layer outputs an output result. The division processing layer 5x and the multiplication processing layer 5y are connected to second vector data 10, which is second vector data including the features of the first vector data of complex numbers, which is input data during training.

[0081] The input vector 5a, the second vector data 10, etc. are complex number data, but for example, the real part data and the imaginary part data may be stored separately as separate data.

[0082] Next, the processing in the nth layer will be described. First, data output from the (n-1)th layer is input as input vector 5a. Typically, data output from the (n-1)th division processing layer 5a is output as input vector 5a. Input vector 5a is, for example, vector data of complex numbers. Next, linear layer 5b performs linear operations on the input data input to the nth layer, i.e., input vector 5a.

[0083] Next, the division processing layer 5x divides the output result of the linear layer 5b by complex second vector data 10 containing features of the training input data. That is, the division processing layer 5x performs a process of dividing the input complex first vector data by complex second vector data containing features of the first vector data. Here, the second vector data 10 may be a map related to a sensitivity map, a B0 (static magnetic field) map, a B1 (radio frequency magnetic field) map, a noise map, or the like. That is, examples of the second vector data 10 include maps that represent features such as a sensitivity map, a B0 map, a B1 map, a noise map, and are expressed in complex numbers. However, examples of the second vector data 10 may directly use a sensitivity map, a B0 map, a B1 map, a noise map, or the like. Complex vector data is generally selected as the second vector data 10.

[0084] Here, the division processing layer 5x performs scaling by dividing the second vector data 10. By performing such processing, the phase component contained in the second vector data 10 is canceled from the first vector data of the input data, the phase component of the first vector data relative to the phase component contained in the second vector data is extracted, and learning can be performed on that phase component. This improves the learning stability of the neural network 7 with respect to phase. Similarly, by dividing the second vector data 10, the relative strength of the amplitude component of the first vector data relative to the amplitude component of the second vector data is extracted, and learning can be performed on that relative strength. This improves the learning stability of input data during learning.

[0085] Next, the nonlinear layer 5c performs activation by applying a nonlinear function to the output result of the division processing layer 5x. An example of the nonlinear function used here is a function obtained by extending the ReLu function to complex numbers.

[0086] Next, the multiplication processing layer 5y multiplies the output result of the nonlinear layer 5c by the second vector data to calculate a complex feature map. In this way, the data scaled in the division processing layer is restored by the multiplication processing layer. The multiplication processing layer 5y outputs the complex feature map resulting from the multiplication as output data of the nth layer and inputs it to the (n+1)th layer as an input vector 6a. Note that the embodiment is not limited to the above example. For example, a BatchNormalization layer may be provided between the division processing layer 5x and the nonlinear layer 5c, or a Dropout layer may be provided after the nonlinear layer 5c or after the multiplication processing layer 5y.

[0087] In this way, the data processing device 100 according to the first embodiment performs scaling on input data using the division processing layer, which improves learning stability for the phase and amplitude of input data during learning, leading to improved image quality of the output image.

[0088] (Second embodiment) In the above embodiment, the case where the second vector data 10, which is data for scaling, is provided in advance has been described. However, the embodiment is not limited to this. In the second embodiment, a case where the second vector data is obtained as an output result of another neural network will be described.

[0089] An example of such a configuration is shown in Figure 7. In the second embodiment, there is a second neural network 20 that is different from the neural network 7 that includes the intermediate layer 5, the intermediate layer 6, etc. Here, the neural network 20 may or may not include a learning parameter, but is preferably a linear network that does not include a nonlinear operation or a bias term.

[0090] Here, an input vector 5a is input to the input side of neural network 20, and second vector data 10 is output from the output side of neural network 20. That is, in the second embodiment, input vector 5a is input to second neural network 20, and second vector data 10, which is data for scaling, is generated as the output result of second neural network 20. That is, in the second embodiment, second vector data 10 is output data obtained by inputting input data into second neural network 20, which is different from neural network 7.

[0091] The second neural network 20 may be the same neural network for each layer of the neural network 7, or may be a neural network whose internal parameters change depending on each layer.

[0092] In this way, in the second embodiment, scaling is performed based on the second vector data generated by another neural network, which allows flexible scaling according to the properties of the input data and improves image quality.

[0093] (Third embodiment) The embodiments are not limited to the above examples, and various variations of the embodiments are possible. As an example, in the first embodiment, the division processing unit 5x is arranged after the linear layer 5b, but the embodiments are not limited to this. In the third embodiment, the division processing unit 5x is arranged before the linear layer 5b, as shown in FIG. 8. That is, in the third embodiment, the linear layer 5b is arranged after the division processing layer 5x and before the multiplication processing layer 5y.

[0094] That is, each layer of the neural network 7 of the trained model according to the third embodiment is The neural network includes a division processing layer 5x to which an input vector 5a is input, a linear layer 5b arranged after the division processing layer 5x, a nonlinear layer 5c arranged after the linear layer 5b, and a multiplication processing layer 5y arranged after the nonlinear layer 5c. The division processing layer 5x divides the input data input to the input layer 5a of each layer by second vector data. The linear layer 5b performs linear operations on the output result of the division processing layer 5x. The nonlinear layer 5c performs nonlinear operations on the output result of the linear layer 5b. The multiplication processing layer 5y multiplies the output result of the nonlinear layer 5c by second vector data 10 and outputs the multiplication result as the layer's output data.

[0095] In this way, in the third embodiment, by arranging the division processing layer 5x before the linear layer 5b, it may be possible to improve the image quality of the output image.

[0096] (Fourth embodiment) The embodiment is not limited to the above example. In the first embodiment, the division and multiplication by the second vector data are performed in each layer. However, the embodiment is not limited to this. The division and multiplication by the second vector data 10 may be performed only once in the first layer and only once in the last layer.

[0097] Such a situation is shown in Figure 9. In Figure 9, hidden layer 3 represents the first layer among the layers of neural network 7, and hidden layer 8 represents the last layer among the layers of neural network 7. Hidden layer 3 includes a linear layer 3b to which input vector 3a is input, a division processing layer 3x, and a nonlinear layer 3c, while hidden layer 8 includes a linear layer 8b to which input vector 8a is input, a nonlinear function 8c, and a multiplication processing layer 8y. In other words, division processing layer 3x is located at the first layer among the layers of neural network 7, and multiplication processing layer 8y is located at the last layer among the layers of the neural network.

[0098] In this way, in the fourth embodiment, the division processing layer 3x is arranged in the first layer, and the multiplication processing layer 8y is arranged in the last layer, and each process is performed only once, thereby reducing the calculation load.

[0099] (Fifth embodiment) In the fifth embodiment, a case will be described in which second vector data, which is data for scaling, is generated based on third vector data, which is another data. Fig. 10 is a diagram showing an example of the configuration of a neural network 7 according to the fifth embodiment.

[0100] The third data 30 is data for generating second vector data 10a, 10b, etc., which are data for scaling. The third vector data 30 is, for example, data such as a map related to sensitivity, B0, or noise. The third vector data 30 may be the first vector data, or data generated using multiple data from the first vector data, sensitivity, B0, B1, or a map related to noise. The neural networks 20a, 20b are themselves neural networks that output data in response to input of predetermined data. For example, the neural network 20a outputs the second vector data 10a based on the input vector 5a and the third vector data. The neural network 20b outputs the second vector data 10b based on the input vector 6a, the second vector data 10a, and input from the neural network 20a.

[0101] Furthermore, the division processing layer 5x divides the output result from the linear layer 5b by the second vector data 10a and outputs the division result to the nonlinear layer 5c. The multiplication processing layer 5y multiplies the output result from the nonlinear layer 5c by the second vector data 10b and outputs the multiplication result as the input vector 6a.

[0102] Similarly, the division processing layer 6x divides the output result from the linear layer 6b by the second vector data 10b and outputs the division result to the nonlinear layer 6c. The multiplication processing layer 6y multiplies the output result from the nonlinear layer 6c by the second vector data 10b and outputs the multiplication result.

[0103] As described above, in the fifth embodiment, the second vector data is output data obtained by inputting input data obtained from the third vector data 30 into a second neural network different from the neural network 7. In this way, by making the second vector data, which is data to be scaled, the output result of the neural network for each layer, the second vector data can be made variable for each layer, allowing for flexible scaling.

[0104] According to at least one of the embodiments described above, image quality can be improved.

[0105] With respect to the above embodiment, the following supplementary notes are disclosed as one aspect and optional features of the invention.

[0106] (Appendix 1) A data processing device provided in one aspect of the present invention includes a processing circuit that processes data using a trained model consisting of a neural network having a division processing layer that performs processing to divide input first vector data of complex numbers by second vector data of complex numbers that includes features of the first vector data, a nonlinear layer that is located after the division processing layer and performs nonlinear arithmetic processing, and a multiplication processing layer that is located after the nonlinear layer and performs processing to multiply input data by the second vector data.

[0107] (Appendix 2) The processing circuitry may output complex vector data.

[0108] (Appendix 3) The neural network may include a linear layer that performs linear operations.

[0109] (Appendix 4) The linear layer may be disposed before the division processing layer.

[0110] (Appendix 5) The linear layer may be arranged after the division processing layer and before the multiplication processing layer.

[0111] (Appendix 6) The second vector data may be a map related to sensitivity, B0, B1 or noise.

[0112] (Appendix 7) The second vector data may be data obtained by inputting input data into a second neural network different from the neural network.

[0113] (Appendix 8) the division processing layer is arranged in the frontmost layer of the layers included in the neural network, The multiplication processing layer may be arranged as a final layer among layers included in the neural network.

[0114] (Appendix 9) The second vector data may be output data obtained by inputting input data obtained from third vector data into a second neural network different from the neural network.

[0115] (Appendix 10) The third vector data may be a map related to the first vector data, sensitivity, B0, B1 or noise, or data generated using a plurality of data from the first vector data, sensitivity, B0, B1 or maps related to noise.

[0116] (Appendix 11) The trained model is A linear layer that performs linear operations on input data, the division processing layer divides an output result of the linear layer by the second vector data; the nonlinear layer performs the nonlinear arithmetic processing on the output result of the division processing layer; The multiplication processing layer may multiply the output result of the nonlinear layer by the second vector data and output the multiplication result as output data of the layer.

[0117] (Appendix 12) The trained model is A linear layer that performs linear operations on input data, the division processing layer divides input data by the second vector data; the linear layer performs the linear operation on the output result of the division processing layer; the nonlinear layer performs the nonlinear arithmetic processing on the output result of the linear layer; The multiplication processing layer may multiply the output result of the nonlinear layer by the second vector data and output the multiplication result as output data of the layer.

[0118] (Appendix 13) A magnetic resonance imaging apparatus according to one aspect of the present invention includes a processing circuit that processes data using a trained model consisting of a neural network having a division processing layer that divides input first vector data of complex numbers by second vector data of complex numbers that includes features of the first vector data, a nonlinear layer that is located after the division processing layer and performs nonlinear arithmetic processing, and a multiplication processing layer that is located after the nonlinear layer and multiplies input data by the second vector data.

[0119] (Appendix 14) A data processing method provided in one aspect of the present invention performs data processing using a trained model consisting of a neural network having a division processing layer that performs processing to divide input first vector data of complex numbers by second vector data of complex numbers that includes features of the first vector data, a nonlinear layer that is located after the division processing layer and performs nonlinear arithmetic processing, and a multiplication processing layer that is located after the nonlinear layer and performs processing to multiply input data by the second vector data.

[0120] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0121] 110 Processing circuit 132 memory 134 Input Device 135 Display

Claims

1. a division processing layer that performs a process of dividing input first vector data of complex numbers by second vector data of complex numbers including features of the first vector data; a nonlinear layer that is disposed after the division processing layer and performs nonlinear arithmetic processing; a multiplication processing layer that is arranged after the nonlinear layer and performs a process of multiplying input data by the second vector data,

2. 2. The data processing apparatus according to claim 1, wherein said processing circuitry outputs vector data of complex numbers.

3. The data processing apparatus according to claim 1 , wherein the neural network comprises a linear layer that performs linear operations.

4. The data processing device according to claim 3 , wherein the linear layer is arranged before the division processing layer.

5. The data processing device according to claim 3 , wherein the linear layer is arranged after the division processing layer and before the multiplication processing layer.

6. the division processing layer is arranged in the frontmost layer of the layers included in the neural network, The data processing device according to claim 1 , wherein the multiplication processing layer is arranged in a final layer among layers included in the neural network.

7. 7. The data processing apparatus according to claim 1, wherein the second vector data is a map obtained by a magnetic resonance imaging apparatus, and is a map related to sensitivity, B0, B1 or noise.

8. 8. The data processing device according to claim 1, wherein the second vector data is data obtained by inputting input data into a second neural network different from the neural network.

9. 2. The data processing device according to claim 1, wherein the second vector data is output data obtained by inputting input data obtained from third vector data into a second neural network different from the neural network.

10. the third vector data is data generated using a map related to the first vector data, sensitivity, B0, B1, or noise, or a plurality of data among the first vector data, sensitivity, B0, B1, or noise, 10. The data processing apparatus according to claim 9, wherein the map relating to sensitivity, B0, B1 or noise is a map obtained by a magnetic resonance imaging apparatus.

11. A data processing device as described in claim 1, wherein the first vector data is a medical image or medical image data obtained by a medical image diagnostic device.

12. A data processing device as described in claim 11, wherein the medical image is a magnetic resonance image obtained by a magnetic resonance imaging device or an ultrasound image obtained by an ultrasound diagnostic device.

13. A data processing device as described in claim 11, wherein the medical image data is image data, projection data, intermediate data or raw data at a stage prior to the generation of a medical image by a medical image diagnostic device.

14. A data processing device as described in claim 1, wherein the data input to the multiplication processing layer is data that has passed through the nonlinear layer.

15. A data processing device as described in claim 1, wherein data processed in the division processing layer is input to the nonlinear layer, and data processed in the nonlinear layer is input to the multiplication processing layer.

16. a division processing layer that performs a process of dividing input first vector data of complex numbers by second vector data of complex numbers including features of the first vector data; a nonlinear layer that is disposed after the division processing layer and performs nonlinear arithmetic processing; a multiplication processing layer that is arranged after the nonlinear layer and performs a process of multiplying input data by the second vector data,

17. a division processing layer that performs a process of dividing input first vector data of complex numbers by second vector data of complex numbers including features of the first vector data; a nonlinear layer that is disposed after the division processing layer and performs nonlinear arithmetic processing; a multiplication processing layer that is arranged after the nonlinear layer and performs a process of multiplying input data by the second vector data,

Images