Multidimensional arithmetic unit and medical imaging diagnostic device

Abstract

To perform machine learning efficiently.SOLUTION: A hypercomplex operation device according to an embodiment includes an acquisition unit, an input unit, and an application unit. The acquisition unit acquires data including a hypercomplex number. The input unit inputs the parametric function in the data in which a function form for a first component differs from a function form for a second component different from the first component. The application unit inputs the data including a hypercomplex number and applies it to the parametric function, thereby outputting output data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The embodiments disclosed in the present specification and drawings relate to a multi-dimensional arithmetic device and a medical image diagnostic apparatus. [Background Art]

[0002] Real numbers are standardly used in machine learning using neural networks, because real numbers have completeness, algebraically have the field structure that allows addition, subtraction, multiplication and division, and are convenient for describing continuous changes of various quantities.

[0003] However, number systems that are closed to certain types of operations and allow free operations within the system are not limited to real numbers. As examples of such number systems, multi-dimensional numbers such as complex numbers and quaternions are known. Here, multi-dimensional numbers are not only merely mathematically interesting, but also have many useful cases in signal processing. For example, in medical image diagnostic apparatuses such as ultrasonic diagnostic apparatuses and magnetic resonance imaging apparatuses, since many sine waves and Fourier transforms are handled, a large amount of signal processing using complex numbers is performed. In addition, quaternions are useful for describing free rotation in three-dimensional space. Besides medical image processing, for example, for audio signals, signals analyzed by short-time Fourier transform are complex numbers, so signal processing using complex numbers is widely used. Also, for example, complex numbers are used in radar systems.

[0004] Therefore, performing processing using a neural network based on multi-dimensional numbers can be expected to contribute to improving the efficiency of signal processing. [Prior Art Documents] [Non-Patent Documents]

[0005] [Non-Patent Document 1] Chiheb Trabelsi et al、Deep Complex Networks、 arXiv preprint 1705.09792 、https: / / arxiv.org / pdf / 1705.09292.pdf SUMMARY OF THE INVENTION PROBLEM TO BE SOLVED BY THE INVENTION

[0006] One of the problems to be solved by the embodiments disclosed in the present specification and drawings is to efficiently perform machine learning. However, the problem to be solved by the embodiments disclosed in the present specification and drawings is not limited to the above problem. Problems corresponding to the respective effects achieved by the respective configurations shown in the embodiments described below can also be positioned as other problems. MEANS FOR SOLVING THE PROBLEM

[0007] A multi-variable arithmetic device according to an embodiment includes an acquisition unit, an input unit, and an application unit. The acquisition unit acquires data containing multiple variables. The input unit inputs a parametric function in which, among the data, a function shape for a first component is different from a function shape for a second component different from the first component. The application unit receives the data containing multiple variables, applies the data to the parametric function, and outputs output data. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] [Figure 1] FIG. 1 is a diagram showing the configuration of the data processing device according to the embodiment. [Figure 2] FIG. 2 is a diagram showing the configuration of the ultrasonic diagnostic apparatus according to the embodiment. [Figure 3] FIG. 3 is a diagram showing the configuration of the magnetic resonance imaging apparatus according to the embodiment. [Figure 4] FIG. 4 is a diagram explaining a neural network according to the embodiment. [Figure 5] FIG. 5 is a diagram explaining processing according to the embodiment. [Figure 6] FIG. 6 is a diagram explaining processing according to the embodiment. [Figure 7] FIG. 7 is a diagram explaining an example of the configuration of the neural network according to the embodiment. [Figure 8]Figure 8 illustrates an example of the configuration of a neural network according to the embodiment. [Figure 9] Figure 9 illustrates an example of the configuration of a neural network according to the embodiment. [Figure 10] Figure 10 is a diagram illustrating the process according to the embodiment. [Figure 11] Figure 11 is a diagram illustrating the process according to the embodiment. [Figure 12] Figure 12 is a diagram illustrating the process according to the embodiment. [Figure 13] Figure 13 is a diagram illustrating the process according to the embodiment. [Figure 14] Figure 14 is a flowchart illustrating the processing flow performed by the data processing device according to the embodiment. [Figure 15] Figure 15 illustrates an example of the configuration of a neural network according to the embodiment. [Figure 16] Figure 16 is a diagram illustrating an example of the configuration of a neural network according to the embodiment. [Figure 17] Figure 17 is a diagram illustrating an example of the configuration of a neural network according to the embodiment. [Modes for carrying out the invention]

[0009] Hereinafter, embodiments of the multidimensional arithmetic device and medical image diagnostic device according to the embodiment will be described in detail with reference to the drawings. The data processing device 100 in Figure 1 is an example of the multidimensional arithmetic device according to the embodiment.

[0010] The data processing device 100 is a device that generates data using machine learning. For example, the data processing device 100 is connected to various medical imaging diagnostic devices such as the ultrasound diagnostic device shown in Figure 2 and the magnetic resonance imaging device shown in Figure 3, and performs tasks such as processing signals received from the medical imaging diagnostic device, generating trained models, and executing trained models. The medical image processing device 100 comprises a processing circuit 110, a memory 132, an input device 134, and a display 135. The processing circuit 110 includes a training data creation function 110c, a learning function 110b, an interface function 110c, a control function 110d, an application function 110e, and an acquisition function 110f.

[0011] In this embodiment, each processing function performed by the training data creation function 110c, learning function 110b, interface function 110c, control function 110d, application function 110e, and acquisition function 110f, as well as the trained model (e.g., a neural network), are stored in memory 132 in the form of a program that can be executed by a computer. The processing circuit 110 is a processor that reads the program from memory 132 and executes it to realize the function corresponding to each program. In other words, the processing circuit 110, when it has read each program, will have the functions shown in the processing circuit 110 in Figure 1. Furthermore, the processing circuit 110, when it has read the program corresponding to the trained model (neural network), can perform processing according to that trained model. In Figure 1, the functions of the processing circuit 110 are described as being realized by a single processing circuit, but the processing circuit 110 may be configured by combining multiple independent processors, and each processor may realize the functions by executing a program. In other words, each of the above functions may be configured as a program, and one processing circuit may execute each program. Furthermore, a single processing circuit may implement two or more of the functions of the processing circuit 110. As another example, a specific function may be implemented in a dedicated, independent program execution circuit.

[0012] In Figure 1, the processing circuit 110, training data creation function 110a, learning function 110b, interface function 110c, control function 110d, application function 110e, and acquisition function 110f are examples of a processing unit, creation unit, input unit (learning unit), reception unit, control unit, application unit, and acquisition unit, respectively.

[0013] In the above explanation, the term "processor" refers to circuits such as a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), an Application Specific Integrated Circuit (ASIC), or a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). The processor functions by reading and executing programs stored in memory 132.

[0014] Alternatively, instead of saving the program in memory 132, the program may be directly incorporated into the processor's circuitry. In this case, the processor performs its functions by reading and executing the program incorporated into the circuitry. Therefore, for example, instead of saving the trained model in memory 132, the program related to the trained model may be directly incorporated into the processor's circuitry.

[0015] Furthermore, if the processing circuit 110 is incorporated into various medical imaging diagnostic devices, or if it performs processing in conjunction with various medical imaging diagnostic devices, it may have a function to execute related processing together.

[0016] The processing circuit 110 acquires data and images for image generation by the application function 110e from the memory 132 via the interface function 110c.

[0017] The processing circuit 110 generates training data for learning based on the data and images acquired by the interface function 110c, using the training data generation function 110a. Details of the processing of the training data generation function 110a and the learning function 110b will be described later.

[0018] The processing circuit 110 uses the training data generated by the training data creation function 110a to perform training using the learning function 110b and generate a trained model.

[0019] The processing circuit 110 controls the entire processing of the data processing device 100 through its control function 110d. Specifically, the processing circuit 110 controls the processing of the processing circuit 110 based on various setting requests input from the operator via the input device 134, and various control programs and data read from the memory 132, through its control function 110d.

[0020] Furthermore, the processing circuit 110 generates an image based on the results of processing performed using the training data generation function 110a and the learning function 110b, via the application function 110e. Additionally, the processing circuit 110 applies the trained model generated by the learning function 110b to the input image via the application function 110e, and generates an image based on the results of applying the trained model.

[0021] Memory 132 consists of semiconductor memory elements such as RAM (Random Access Memory) and flash memory, a hard disk, and an optical disc. Memory 132 is a memory that stores data such as display image data and training image data generated by the processing circuit 110.

[0022] Memory 132 stores various data, such as control programs for image processing and display processing, as needed.

[0023] The input device 134 receives various instructions and information inputs from the operator. The input device 134 is, for example, a pointing device such as a mouse or trackball, a selection device such as a mode switch, or an input device such as a keyboard.

[0024] The display 135 displays a GUI (Graphical User Interface) for accepting input of imaging conditions, or images generated by the control function 110d, under the control of the control function 110y, etc. The display 135 is a display device such as a liquid crystal display. The display 135 is an example of a display unit. The display 135 has a mouse, keyboard, buttons, panel switches, touch command screen, foot switch, trackball, joystick, etc.

[0025] Next, we will describe an example of the configuration of a medical image processing device using the data processing device 100, which is a multidimensional arithmetic device according to the embodiment.

[0026] Figure 2 shows an example of the configuration of an ultrasound diagnostic apparatus 10 incorporating a multidimensional arithmetic device according to the embodiment as a data processing device 100. The ultrasound diagnostic apparatus according to the embodiment includes an ultrasound probe 5 and an ultrasound diagnostic apparatus body 10. The ultrasound diagnostic apparatus body 10 includes a transmission circuit 9, a reception circuit 11, and the above-mentioned data processing device 100.

[0027] The ultrasonic probe 5 has multiple piezoelectric transducers, which generate ultrasound based on drive signals supplied from a transmission circuit 9 in the ultrasound diagnostic device body 10, which will be described later. The multiple piezoelectric transducers of the ultrasonic probe 5 also receive reflected waves from the subject P and convert them into electrical signals (reflected wave signals). The ultrasonic probe 5 also has a matching layer provided on the piezoelectric transducers and a backing material to prevent the propagation of ultrasound backward from the piezoelectric transducers. The ultrasonic probe 5 is detachably connected to the ultrasound diagnostic device body 10. The ultrasonic probe 5 is also an example of a scanning unit.

[0028] When ultrasound is transmitted from the ultrasound probe 5 to the subject P, the transmitted ultrasound is reflected one after another by discontinuities in acoustic impedance within the subject P's internal tissues. The reflected waves are received by multiple piezoelectric transducers on the ultrasound probe 5 and converted into reflected wave signals. The amplitude of the reflected wave signal depends on the difference in acoustic impedance at the discontinuities where the ultrasound is reflected. If the transmitted ultrasound pulse is reflected by a moving blood flow or the surface of the heart wall, the reflected wave signal undergoes a frequency shift due to the Doppler effect, depending on the velocity component of the moving object relative to the ultrasound transmission direction.

[0029] The ultrasound diagnostic device body 10 is a device that generates ultrasound image data based on reflected wave signals received from the ultrasound probe 5. The ultrasound diagnostic device body 10 is capable of generating two-dimensional ultrasound image data based on two-dimensional reflected wave signals, and is capable of generating three-dimensional ultrasound image data based on three-dimensional reflected wave signals. However, the embodiment is also applicable even when the ultrasound diagnostic device 10 is a device dedicated to two-dimensional data.

[0030] As illustrated in Figure 2, the ultrasound diagnostic device 10 includes a transmitting circuit 9, a receiving circuit 11, and a medical image processing device 100.

[0031] The transmitting circuit 9 and the receiving circuit 11 control the ultrasonic transmission and reception performed by the ultrasonic probe 5 based on instructions from a data processing device 110 with control functions. The transmitting circuit 9 includes a pulse generator, a transmission delay unit, a pulser, etc., and supplies a drive signal to the ultrasonic probe 5. The pulse generator repeatedly generates rate pulses to form the transmitted ultrasonic waves at a predetermined pulse repetition frequency (PRF). The transmission delay unit provides a delay time for each piezoelectric vibrator necessary to focus the ultrasonic waves generated from the ultrasonic probe 5 into a beam and determine the transmission directivity, for each rate pulse generated by the pulse generator. The pulser applies a drive signal (drive pulse) to the ultrasonic probe 5 at a timing based on the rate pulse.

[0032] In other words, the transmission delay unit arbitrarily adjusts the transmission direction of the ultrasonic waves transmitted from the piezoelectric transducer surface by changing the delay time applied to each rate pulse. Furthermore, the transmission delay unit controls the position of the focal point (transmission focus) in the depth direction of ultrasonic transmission by changing the delay time applied to each rate pulse.

[0033] Furthermore, the transmitting circuit 9 has the function of instantaneously changing the transmitting frequency, transmitting drive voltage, etc., in order to execute a predetermined scan sequence based on instructions from the processing circuit 110, which will be described later. In particular, the change in the transmitting drive voltage is achieved by a linear amplifier type oscillator circuit that can switch its value instantaneously, or by a mechanism that electrically switches multiple power supply units.

[0034] The receiving circuit 11 also includes an amplifier circuit, an A / D (Analog / Digital) converter, a receiving delay circuit, an adder, a quadrature detection circuit, etc., and performs various processing on the reflected wave signal received from the ultrasonic probe 5 to generate a received signal (reflected wave data). The amplifier circuit amplifies the reflected wave signal for each channel and performs gain correction processing. The A / D converter performs A / D conversion of the gain-corrected reflected wave signal. The receiving delay circuit gives the digital data the receiving delay time necessary to determine the receiving directivity. The adder performs summing processing on the reflected wave signal to which the receiving delay time has been given by the receiving delay circuit. Through the summing processing of the adder, the reflected component from the direction corresponding to the receiving directivity of the reflected wave signal is emphasized. The quadrature detection circuit then converts the output signal of the adder into a baseband in-phase signal (I signal, I: In-phase) and a quadrature signal (Q signal, Q: Quadrature-phase). The quadrature detection circuit then transmits the I signal and Q signal (hereinafter referred to as IQ signal) as the received signal (reflected wave data) to the processing circuit 110. Alternatively, the quadrature detection circuit may convert the adder's output signal into an RF (Radio Frequency) signal and then transmit it to the processing circuit 110. The IQ signal and the RF signal are received signals that contain phase information.

[0035] When scanning a two-dimensional region within the subject P, the transmitting circuit 9 causes the ultrasonic probe 5 to transmit an ultrasonic beam for scanning the two-dimensional region. The receiving circuit 11 then generates a two-dimensional received signal from the two-dimensional reflected wave signal received from the ultrasonic probe 5. When scanning a three-dimensional region within the subject P, the transmitting circuit 9 causes the ultrasonic probe 5 to transmit an ultrasonic beam for scanning the three-dimensional region. The receiving circuit 11 then generates a three-dimensional received signal from the three-dimensional reflected wave signal received from the ultrasonic probe 5. The receiving circuit 11 generates a received signal based on the reflected wave signal and transmits the generated received signal to the processing circuit 110.

[0036] The transmitting circuit 9 causes the ultrasonic probe 5 to transmit an ultrasonic beam from a predetermined transmitting position (transmitting scan line). The receiving circuit 11 receives the signal from the ultrasonic probe 5 at a predetermined receiving position (receiving scan line) due to the reflected wave of the ultrasonic beam transmitted by the transmitting circuit 9. If parallel simultaneous reception is not performed, the transmitting scan line and the receiving scan line will be the same scan line. On the other hand, if parallel simultaneous reception is performed, when the transmitting circuit 9 transmits one ultrasonic beam to the ultrasonic probe 5 on one transmitting scan line, the receiving circuit 11 simultaneously receives the signal from the reflected wave originating from the ultrasonic beam transmitted by the transmitting circuit 9 to the ultrasonic probe 1 as multiple receiving beams at multiple predetermined receiving positions (receiving scan lines) through the ultrasonic probe 5.

[0037] The data processing unit 100 is connected to the transmitting circuit 9 and the receiving circuit 11, and in addition to the functions already shown in Figure 1, it processes the signal received from the receiving circuit 11, controls the transmitting circuit 9, generates a trained model, executes the trained model, and performs various image processing. The processing circuit 110, in addition to the functions already shown in Figure 1, includes B-mode processing, Doppler processing, generation functions, etc. Below, we will describe configurations that the data processing unit 100 incorporated into the ultrasound diagnostic device 10 may have in addition to the configuration already shown in Figure 1.

[0038] In this embodiment, the B-mode processing function, Doppler processing function, and generation function, along with the learned models, are stored in memory 132 in the form of programs executable by the computer. The processing circuit 110 is a processor that reads programs from memory 132 and executes them to realize the functions corresponding to each program. In other words, the processing circuit 110, in the state where each program has been read, will have these functions.

[0039] The B-mode processing function and Doppler processing function are examples of B-mode processing units and Doppler processing units.

[0040] The processing circuit 110 performs various signal processing operations on the received signal received from the receiving circuit 11.

[0041] The processing circuit 110 receives data from the receiving circuit 11 using its B-mode processing function, and performs logarithmic amplification, envelope detection, logarithmic compression, etc., to generate data (B-mode data) in which the signal strength is expressed in terms of brightness.

[0042] Furthermore, the processing circuit 110 uses Doppler processing to perform frequency analysis on the received signal (reflected wave data) received from the receiving circuit 11 to obtain velocity information and generate data (Doppler data) that extracts moving object information such as velocity, dispersion, and power at multiple points due to the Doppler effect.

[0043] Furthermore, the B-mode processing function and Doppler processing function can process both 2D and 3D reflected wave data.

[0044] Furthermore, the processing circuit 110 controls the entire processing of the ultrasound diagnostic device through the control function 110d. Specifically, the processing circuit 110 controls the processing of the transmission circuit 9, the reception circuit 11, and the processing circuit 110 based on various setting requests input from the operator via the input device 134, and various control programs and data read from the memory 132, through the control function 110d. In addition, the processing circuit 110 controls the display of ultrasound image data stored in the memory 132 on the display 135 through the control function 110d.

[0045] Furthermore, the processing circuit 110 generates ultrasonic image data from the data generated by the B-mode processing function and the Doppler processing function using a generation function (not shown). The processing circuit 110 generates two-dimensional B-mode image data, in which the intensity of reflected waves is represented by brightness, from the two-dimensional B-mode data generated by the B-mode processing function using a generation function. The processing circuit 110 also generates two-dimensional Doppler image data representing moving object information from the two-dimensional Doppler data generated by the Doppler processing function 110b using a generation function. The two-dimensional Doppler image data is velocity image data, dispersion image data, power image data, or image data combining these.

[0046] Furthermore, the processing circuit 110, through its generation function, converts the scan line signal sequence of the ultrasonic scan into a scan line signal sequence of a video format, such as that used in televisions, to generate ultrasonic image data for display. In addition to scan conversion, the processing circuit 110, through its generation function, also performs various image processing tasks, such as image processing that regenerates an average brightness image using multiple image frames after scan conversion (smoothing process), and image processing that uses a differential filter within the image (edge ​​enhancement process). Furthermore, the processing circuit 110, through its generation function, performs various rendering processes on the volume data in order to generate two-dimensional image data for displaying the volume data on the display 135.

[0047] Memory 132 can also store data generated during B-mode processing and Doppler processing. The B-mode data and Doppler data stored in memory 132 can be retrieved by the operator after a diagnosis, for example, and become ultrasound image data for display via the processing circuit 110. Memory 132 can also store the received signal (reflected wave data) output by the receiving circuit 11.

[0048] In addition, memory 132 stores control programs for ultrasound transmission and reception, image processing and display processing as needed, as well as various data such as diagnostic information (e.g., patient ID, physician's findings, etc.), diagnostic protocols, and various body marks.

[0049] Figure 3 shows an example configuration of a magnetic resonance imaging apparatus 200 that incorporates a multidimensional arithmetic device according to the embodiment as a data processing device 100.

[0050] As shown in Figure 3, the magnetic resonance imaging apparatus 200 comprises a static magnetic field magnet 201, a static magnetic field power supply (not shown), a gradient magnetic field coil 203, a gradient magnetic field power supply 204, a bed 205, a bed control circuit 206, a transmitting coil 207, a transmitting circuit 208, a receiving coil 209, a receiving circuit 210, a sequence control circuit 220 (sequence control unit), and a data acquisition device 100. Note that the magnetic resonance imaging apparatus 200 does not include a subject P (e.g., a human body). Also, the configuration shown in Figure 3 is merely an example.

[0051] The static magnetic field magnet 201 is a hollow, substantially cylindrical magnet that generates a static magnetic field in its internal space. The static magnetic field magnet 201 is, for example, a superconducting magnet and is excited by the supply of current from the static magnetic field power supply. The static magnetic field power supply supplies current to the static magnetic field magnet 201. As an alternative example, the static magnetic field magnet 201 may be a permanent magnet, in which case the magnetic resonance imaging apparatus 200 does not need to have a static magnetic field power supply. Also, the static magnetic field power supply may be provided separately from the magnetic resonance imaging apparatus 100.

[0052] The gradient coil 203 is a hollow, substantially cylindrical coil placed inside the static magnetic field magnet 201. The gradient coil 203 is formed by combining three coils corresponding to the mutually orthogonal X, Y, and Z axes. These three coils receive current individually from the gradient power supply 204, generating gradient magnetic fields whose magnetic field strength changes along the X, Y, and Z axes. The gradient magnetic fields in the X, Y, and Z axes generated by the gradient coil 203 are, for example, a slicing gradient magnetic field Gs, a phase encoding gradient magnetic field Ge, and a readout gradient magnetic field Gr. The gradient power supply 204 supplies current to the gradient coil 203.

[0053] The bed 205 is equipped with a top plate 205a on which the subject P is placed, and under the control of the bed control circuit 206, the top plate 205a is inserted into the cavity (imaging port) of the gradient magnetic field coil 203 with the subject P placed on it. Normally, the bed 205 is installed so that its longitudinal direction is parallel to the central axis of the static magnetic field magnet 201. Under the control of the data acquisition device 100, the bed control circuit 206 drives the bed 205 to move the top plate 205a in the longitudinal and vertical directions.

[0054] The transmitting coil 207 is positioned inside the gradient coil 203 and receives RF pulses from the transmitting circuit 208 to generate a high-frequency magnetic field. The transmitting circuit 208 supplies RF pulses to the transmitting coil 207 that correspond to the Larmor frequency, which is determined by the type of atom being targeted and the magnetic field strength.

[0055] The receiving coil 209 is positioned inside the gradient magnetic field coil 203 and receives the magnetic resonance signal (hereinafter referred to as the "MR signal" as needed) emitted from the subject P due to the influence of the high-frequency magnetic field. When the receiving coil 209 receives the magnetic resonance signal, it outputs the received magnetic resonance signal to the receiving circuit 210.

[0056] The transmitting coil 207 and receiving coil 209 described above are merely examples. The system can be constructed by combining one or more coils, such as a coil with only a transmitting function, a coil with only a receiving function, or a coil with both transmitting and receiving functions.

[0057] The receiving circuit 210 detects the magnetic resonance signal output from the receiving coil 209 and generates magnetic resonance data based on the detected magnetic resonance signal. Specifically, the receiving circuit 210 generates magnetic resonance data by digitally converting the magnetic resonance signal output from the receiving coil 209. The receiving circuit 210 also transmits the generated magnetic resonance data to the sequence control circuit 220. The receiving circuit 210 may also be provided on the mounting device side, which includes the static magnetic field magnet 201 and the gradient magnetic field coil 203.

[0058] The sequence control circuit 220 performs imaging of the subject P by driving the gradient power supply 204, the transmitting circuit 208, and the receiving circuit 210 based on sequence information transmitted from the image processing device 230. Here, the sequence information is information that defines the procedure for performing imaging. The sequence information defines the strength and timing of the current supplied by the gradient power supply 204 to the gradient coil 203, the strength and timing of the RF pulse supplied by the transmitting circuit 208 to the transmitting coil 207, and the timing at which the receiving circuit 210 detects the magnetic resonance signal. For example, the sequence control circuit 220 is an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), or an electronic circuit such as a CPU (Central Processing Unit) or MPU (Micro Processing Unit). The sequence control circuit 220 is an example of a scanning unit. Details of the pulse sequence executed by the sequence control circuit 220 will be described later.

[0059] Furthermore, the sequence control circuit 220 drives the gradient magnetic field power supply 204, the transmitting circuit 208, and the receiving circuit 210 to image the subject P. When it receives magnetic resonance data from the receiving circuit 210, it transfers the received magnetic resonance data to the image processing device 230. In addition to the processing described in Figure 1, the data processing device 100 performs overall control of the magnetic resonance imaging device 200.

[0060] Regarding the processing performed by the data processing device 100, other than the processing described in Figure 1, the processing circuit 110 transmits sequence information to the sequence control circuit 220 via the interface function 110c and receives magnetic resonance data from the sequence control circuit 220. Upon receiving the magnetic resonance data, the processing circuit 110, which has the interface function 110c, stores the received magnetic resonance data in the memory 132.

[0061] The magnetic resonance data stored in memory 132 is placed in k-space by the control function 110d. As a result, memory 132 stores k-space data.

[0062] Memory 132 stores magnetic resonance data received by processing circuit 110 having interface function 110c, k-space data arranged in k-space by processing circuit 110 having control function 110d, image data generated by processing circuit 110 having generation function (or application function 110e), and the like.

[0063] The processing circuit 110, through its control function 110d, performs overall control of the magnetic resonance imaging apparatus 120, controlling imaging, image generation, image display, etc. For example, the processing circuit 110 with the control function 110d receives input of imaging conditions (imaging parameters, etc.) via a GUI and generates sequence information according to the received imaging conditions. The processing circuit 110 with the control function 110d also transmits the generated sequence information to the sequence control circuit 220.

[0064] The processing circuit 11 reads k-space data from memory 132 using the generation function (or application function 110e), and generates a magnetic resonance image by applying reconstruction processing such as a Fourier transform to the read k-space data.

[0065] Next, I will explain the background of the embodiment.

[0066] Real numbers are complete and have a field structure that allows for algebraic addition, subtraction, multiplication, and division, making them convenient for describing the continuous changes of various quantities. Therefore, real numbers are standardly used in machine learning using neural networks.

[0067] However, number systems that are closed under certain operations and allow for free operations within them are not limited to real numbers. Examples of such number systems include multidimensional numbers such as complex numbers and quaternions. Complex numbers and quaternions are multidimensional numbers that include real numbers as a proper subset, and the output of operations defined within the multidimensional number system is included as an element of the set of the multidimensional number system, with the exception of some operations, such as division by zero. Among multidimensional numbers, real numbers and complex numbers are commutative under multiplication, but quaternions are noncommutative under multiplication; that is, quaternions constitute a noncommutative algebra.

[0068] These multidimensional numbers are not only mathematically interesting, but also have practical applications in medical image processing. For example, medical imaging devices such as ultrasound and magnetic resonance imaging systems frequently handle sine waves and Fourier transforms, leading to extensive signal processing using complex numbers. Quaternions are also useful for describing free rotation in three-dimensional space. Therefore, it is desirable to perform processing using neural networks that utilize multidimensional numbers in medical image processing.

[0069] In view of this background, the data processing device 100 according to the embodiment acquires data including multidimensional numbers by the acquisition function 110f of the processing circuit 110. Subsequently, by inputting the data including multidimensional numbers into a neural network, which is a parametric function for the data including multidimensional numbers, the processing circuit 110 performs machine learning by the learning function 110b, which acts as an input unit. In other words, the processing circuit 110 inputs data including multidimensional numbers into the neural network by the learning function 110b. Furthermore, the processing circuit 110 outputs output data by inputting data including multidimensional numbers into the parametric function by the application function 110e, which acts as an application unit, and applying it to the parametric function.

[0070] Furthermore, the ultrasound diagnostic apparatus 10 according to the embodiment includes an ultrasound probe 5 and a data processing device 100. The processing circuit 110 acquires the data obtained by the ultrasound probe 5 as data including the multidimensional number using the acquisition function 110f, and outputs the ultrasound diagnostic image as output data using the application function 110e. Furthermore, the magnetic resonance imaging apparatus 200 according to the embodiment includes a sequence control circuit 220 that executes a pulse sequence and a data processing device 100. The processing circuit 110 acquires the data obtained by the pulse sequence as data including the multidimensional number using the acquisition function 110f, and outputs the magnetic resonance image as output data using the application function 110e.

[0071] First, we will explain the general configuration of a neural network using Figure 4.

[0072] Figure 4 shows an example of the interconnection between layers in a neural network used for machine learning by a processing circuit 110 having a learning function 110b. This neural network consists of, for example, an input layer, an output layer, and intermediate layers including convolutional layers and pooling layers, and each layer is composed of multiple nodes (neurons), such as nodes 10 and 11, as shown in Figure 4. Here, each node generates an output by comparing the weighted sum of the inputs of the previous layer with a threshold. As an example, the value z of each node is equal to the value u of the i-th node of the previous layer.i , weight w with the i-th node i Using an activation function f, for example, z = f(Σ i w i u i It is given by ).

[0073] Here, the activation function f can be, for example, a saturated activation function (e.g., a hyperbolic tangent activation function) or a normalized activation function (e.g., a rectified linear unit (ReLU)). Note that Figure 4 is merely one example of a neural network configuration, and various other neural networks are possible as embodiments.

[0074] Next, in this embodiment, the processing circuit 110 inputs data containing multidimensional numbers into the neural network using the learning function 110b. First, we will briefly explain the representation of a neural network where each node has a multidimensional value in a number vector space over the field of real numbers.

[0075] The set of all complex numbers C is, as a set, a two-dimensional vector space R over the field of real numbers. 2 This is equal to . Here, as is well known, addition, scalar multiplication, and multiplication of complex numbers can be defined for complex numbers, and a mapping from one complex number to another can be represented by a 2x2 matrix of real coefficients. This 2x2 matrix of real coefficients has redundancy, and by the definition of complex multiplication, two of the four numbers are independent, and the rest are either the same number or a number with the sign reversed. Therefore, a neural network in which the i-th layer consists of N complex-valued neurons (feature vectors) and the (i+1)-th layer consists of M complex-valued neurons can, if necessary, be represented by a neural network in which the i-th layer consists of 2N real-valued neurons and the (i+1)-th layer consists of 2M real-valued neurons.

[0076] This situation is illustrated in Figure 5. Figure 5 shows the relationship between a neural network whose elements are complex-valued neurons and a neural network whose elements are real-valued neurons. Specifically, Figure 5(a) shows a neural network whose elements are complex-valued nodes 10, 11, 20, and 21, and Figure 5(b) shows the corresponding neural network whose elements are real-valued nodes 10a, 10b, 11a, 11b, 15a, 15b, 16a, and 16b. Here, nodes 10a, 11a, 15a, and 16a are... These are the real parts of nodes 10, 11, 15, and 16, respectively, while nodes 10b, 11b, 15b, and 16b are the imaginary parts of nodes 10, 11, 15, and 16, respectively. In Figure 5(a), the weights of each neuron and the weights between each neuron can take complex values, and such a complex-valued neural network can be represented by a corresponding real-valued neural network.

[0077] Next, as an example of a neural network using multidimensional numbers other than complex numbers, we will explain a neural network using quaternions.

[0078] The set of all quaternions H is, as a set, a four-dimensional vector space R over the field of real numbers. 4 It is equal to quaternions. Addition, scalar multiplication, and multiplication of quaternions can be defined for quaternions. In order to define the multiplication of quaternions, the R of quaternions 4 We must define a basis for the above, which we will denote as 1, i, j, and k. Here, the following equality holds between these basis elements, as shown in equation (1).

[0079]

number

[0080] At this time, an element of a quaternion can be written as a+bi+cj+dk where a, b, c, d are real numbers, and multiplication between quaternions can be defined by using the distributive law and the above-mentioned formula (1). Note that multiplication between quaternions is non-commutative. Furthermore, the entire set of quaternions H is a four-dimensional real number vector space R on the real number field 4 As can also be understood from the equality described above, a mapping from quaternions to quaternions can be represented by a 4-by-4 real coefficient matrix.

[0081] Such a situation is shown in FIG. 6. FIG. 6 shows the relationship between a neural network having quaternion-valued neurons as elements and a neural network having real-valued neurons as elements. Specifically, FIG. 6(a) shows a neural network having quaternion-valued nodes 10, 11, 15, 16 as elements, and FIG. 6(b) shows a corresponding neural network having real-valued nodes 10a, 10b, 10c, 10d, 15a, 15b, 15c, 15d and the like as elements. Here, nodes 10a, 10b, 10c, 10d correspond to quaternion node 10, and R 4 are nodes corresponding to each basis on the above, and here, nodes 15a, 15b, 15c, 15d correspond to quaternion node 15, and R 4 are nodes corresponding to each basis on the above. In FIG. 6(a), each neuron and the weight between each neuron can take quaternion values, and such a quaternion-valued neural network can be represented by a corresponding real-valued neural network.

[0082] Next, in this embodiment, the processing circuit 110, using the learning function 110b, inputs a parametric function as a neural network to the data containing the multidimensional number, where the function shape of the first component (element) and the function shape of the second component (which is different from the first component) are different. Here, examples of the first component (element) and the second component (element) include, for example, the real part and the imaginary part, respectively, when the multidimensional number is a complex number, and the expanded components of the quaternion at each basis that constitutes the quaternion, when the multidimensional number is a quaternion. However, the examples of the first component and the second component are not limited to these.

[0083] The configuration according to the embodiment will be described using Figures 7 and 8. Here, Figure 7 is an example where complex numbers are used as the multidimensional, and Figure 8 is an example where quaternions are used as the multidimensional. The neural network shown in Figure 7 consists of layers of complex number nodes constituting nodes 10, 11, and 12, layers of complex number nodes constituting nodes 80, 81, and 82, and two different neural networks, the first neural network 50 and the second neural network 51, which exist between them. The first neural network 50 is a neural network concerning the real part, and the input and output layers of the first neural network 50 are connected to the real parts of nodes 10, 11, and 12 and the real parts of nodes 80, 81, and 82, respectively. The second neural network 51 is a neural network concerning the imaginary part, and the input and output layers of the second neural network 51 are connected to the imaginary parts of nodes 10, 11, and 12 and the imaginary parts of nodes 80, 81, and 82, respectively.

[0084] Furthermore, the meaning that the first neural network 50 and the second neural network 51 are different neural networks means that the neural network as a parametric function has a different function shape for, for example, the real number component (the first component) and the imaginary number component (the second component). This may also be the case if the number of parameters related to the function shape for the first component and the number of parameters related to the function shape for the second component are different. As an example, the first neural network 50 and the second neural network 51 may be neural networks with different numbers of nodes.

[0085] In this way, by using different neural networks depending on whether the complex number is real or imaginary, for example, neural networks with different numbers of nodes, machine learning can be performed using the number of nodes required for learning the real and imaginary parts respectively, thereby improving the accuracy of machine learning.

[0086] Figure 8 shows an example where quaternions are used as the multidimensional numbers.

[0087] The neural network shown in Figure 8 consists of a layer 60 that extracts each component from a quaternion and converts it into individual components, a layer 70 that synthesizes quaternions from each component, and different neural networks located between them: a first neural network 90 for the first component, a second neural network 91 for the second component, a third neural network 92 for the third component, and a fourth neural network 93 for the fourth component. Of the components converted from the quaternion in layer 60, the first component is input to the first neural network 90, the second component to the second neural network 91, the third component to the third neural network 92, and the fourth component to the fourth neural network 93. Furthermore, the first component output from the first neural network 90, the second component output from the second neural network 91, the third component output from the third neural network 92, and the fourth component output from the fourth neural network 94 are input to layer 70, and as a result, a quaternion is synthesized and output.

[0088] Specifically, if the quaternion z1 input to layer 60 can be written as z1 = a1 + b1i + c1j + d1k using the quaternion basis (1, i, j, k) and real numbers a1, b1, c1, d1, then the first component output to the first neural network 90 will be a1, the second component output to the second neural network 91 will be b1, the third component output to the third neural network 92 will be c1, and the fourth component output to the fourth neural network 93 will be d1. Furthermore, if the first component output from the first neural network 90 and input to layer 70 is a2, the second component output from the second neural network 91 and input to layer 70 is b2, the third component output from the third neural network 92 and input to layer 70 is c2, and the fourth component output from the fourth neural network 93 and input to layer 70 is d2, then layer 70 outputs the quaternion z2 = a2 + b2i + c2j + d2k.

[0089] Furthermore, the first neural network 90, the second neural network 91, the third neural network 92, and the fourth neural network 93 may be different neural networks. That is, the function shapes of these neural networks as parametric functions may differ, for example, in the function shape for the first component, the function shape for the second component, the function shape for the third component, and the function shape for the fourth component. For example, the number of parameters related to the function shape for the first component, the number of parameters related to the function shape for the second component, the number of parameters related to the function shape for the third component, and the number of parameters related to the function shape for the fourth component may be different.

[0090] For example, the first neural network 90, the second neural network 91, the third neural network 92, and the fourth neural network 94 may be neural networks with different numbers of nodes. In particular, since the first component, which is a real number component, and the second to fourth components, which are imaginary number components, often have different properties, the first neural network 90 and the second to fourth neural networks 91 to 94 may be neural networks with different numbers of nodes.

[0091] In the embodiment, the quaternions often consist only of purely imaginary numbers, i.e., quaternions whose first component is 0. In such cases, the processing related to the first neural network 90 may be omitted.

[0092] Furthermore, the components of a quaternion other than the real part, i.e., the second to fourth components of a quaternion, are associated with three-dimensional positions. That is, the quaternion z = a + bi + cj + dk is associated with the three-dimensional position (b, c, d). Also, the three-dimensional position (b, c, d) is associated with the quaternion z = bi + cj + dk. In other words, in Figure 8, layer 60 of the neural network in Figure 8 contains a function that converts quaternions to three-dimensional positions. Similarly, layer 70 of the neural network in Figure 8 contains a function that converts three-dimensional positions to quaternions. In this way, by using the neural network in Figure 8, it is possible to interconnect a quaternion neural network and a three-dimensional position neural network.

[0093] In summary, by using different neural networks depending on each component of a quaternion, for example, neural networks with different numbers of nodes, it is possible to perform machine learning using the number of nodes required for learning each component of the quaternion, such as the real and imaginary parts, thereby improving the accuracy of the machine learning.

[0094] Next, using Figures 9 to 11, we will describe an embodiment in which complex numbers are used as the multidimensional numbers and a neural network is represented in polar coordinates.

[0095] Figure 9 shows the overall structure of such a neural network.

[0096] The neural network shown in Figure 9 consists of a polar coordinate conversion layer 60 that converts complex number nodes 10, 11, and 12 into polar coordinate nodes, a polar coordinate conversion layer 70 that converts the polar coordinate nodes back into complex number nodes 80, 81, and 82, and two different neural networks, the first neural network 52 and the second neural network 53, which exist between them. The first neural network 52 is a neural network relating to the absolute value, which is the first component in the polar coordinate representation of complex numbers, and the input and output layers of the first neural network 52 are connected to the nodes relating to the absolute value among the output nodes of the polar coordinate conversion layer 60 and the nodes relating to the absolute value among the input nodes of the polar coordinate conversion layer 70, respectively. Furthermore, the second neural network 53 is a neural network relating to the argument, which is the second component in the polar coordinate representation of a complex number. The input layer and output layer of the second neural network 53 are connected to the node relating to the argument among the output nodes of the polar coordinate conversion layer 60 and to the node relating to the argument among the input nodes of the polar coordinate deconversion layer 70, respectively.

[0097] Figure 10 shows the configuration of the polar coordinate layer 60. The polar coordinate layer 60 transforms a complex number node 10, which consists of a real part node 10a and an imaginary part node 10b, into an absolute value node 20a and an argument node 20b. For example, when the value of the real part node 10a is a and the value of the imaginary part node 10b is b, the complex number node 10 becomes a + ib, but the polar coordinate layer 60 transforms the value of the absolute value node 20a into (a 2 +b 2 ) 1 / 2 , the value of node 20b of the argument is tan -1 By providing (b / a), the complex number node 10 is converted into the absolute value node 20a and the argument node 20b. Thus, the polar coordinate layer 60 included in the neural network shown in Figure 9 includes a function that converts a complex number displayed in orthogonal coordinates into a complex number displayed in polar coordinates.

[0098] Furthermore, Figure 11 shows the configuration of the polar coordinate deconversion layer 70. The polar coordinate deconversion layer 70 converts the absolute value node 20a and the argument node 20b into a complex number node 80 consisting of a real part node 80a and an imaginary part node 80b. For example, when the value of the absolute value node 20a is r and the value of the argument node 20b is θ, the value of the real part node 20a is rcosθ, the value of the imaginary part node 20b is rsinθ, and the value of the complex number node 80 is r×exp(iθ). The polar coordinate deconversion layer 70 converts nodes represented in polar coordinates into complex number nodes. Thus, the polar coordinate deconversion layer 70 included in the neural network shown in Figure 9 includes a function that converts complex numbers represented in polar coordinates into complex numbers represented in Cartesian coordinates.

[0099] Returning to Figure 9, the first neural network 52 and the second neural network 53 may be different neural networks. That is, the neural network as a parametric function may have different function shapes for, for example, the absolute value component (first component) and the argument component (second component). In other words, for example, the number of parameters related to the function shape for the first component and the number of parameters related to the function shape for the second component may be different functions.

[0100] As an example, as shown in Figure 12, the first neural network 52 and the second neural network 53 may be neural networks with different numbers of nodes. For example, if the first neural network 52 and the second neural network 53 are 2D CNNs (Convolutional Neural Networks), one layer of the first neural network 52 may be a 5x5 network, and one layer of the second neural network 53 may be a 3x3 network.

[0101] In this way, by using different neural networks depending on the absolute value component and argument component of a complex number, for example, neural networks with different numbers of nodes, it is possible to perform machine learning using the number of nodes required for learning the absolute value component and argument component respectively, thereby improving the accuracy of machine learning.

[0102] Furthermore, such a neural network may include, if necessary, a parametric function relating to the argument in the pole-locus representation of a complex number. For example, as shown in Figure 13, layer 91 in the complex neural network may include a parametric function that changes the argument by an angle A, which is a learnable coefficient. Alternatively, layer 90 in the complex neural network may include a parametric function that complexly rotates a complex number node by an angle A. Or, for example, it may include a parametric function that takes an argument as input and outputs the remainder of the input value with respect to 2π radians.

[0103] Next, using Figure 14, we will explain the learning method for neural networks containing multidimensional data as described above. While the following description focuses on learning neural networks with real number components, as already mentioned, the multidimensional number field can be represented on a vector space over the real number field. Therefore, a neural network where each node has a multidimensional value can be represented as a neural network where each node has a real number value. By making appropriate substitutions as needed, the learning method for neural networks with real number components can be extended to neural networks with multidimensional components without losing generality. Furthermore, as can be seen from the fact that, for example, complex number multiplication reduces the coefficients by half compared to simply multiplying two real numbers, it is also possible to use the definition of multidimensional operations to reduce redundant numbers and use that as the target for learning.

[0104] First, the processing circuit 110 acquires data including multidimensional data from a medical diagnostic device, etc., using the acquisition function 110f. Next, the processing circuit 110 creates a training dataset based on the acquired data including multidimensional data using the training data creation function 110a.

[0105] Here, the training dataset D is, for example, D = {{x1, d1}, {x2, d2}, ... {x N d N}}. Here, let i be a natural number, x i is the i-th input data, d i x is the i-th output data (i.e., the correct answer data), and N is the number of training datasets. Note that x is the i-th input data. i and the i-th output data d i This does not necessarily have to be a scalar quantity; it can be a vector quantity, matrix, tensor quantity, or any other data format.

[0106] As an example, the processing circuit 110 acquires multidimensional data related to ultrasound diagnosis from the ultrasound diagnostic device 10 as input data x using the acquisition function 110f, and creates a training dataset D based on the output data d.

[0107] For example, the processing circuit 110 creates a training dataset D for a complex DNN (Deep Neural Network) for performing acoustic clutter removal filtering in beamforming processing, based on complex signal data acquired from the ultrasound diagnostic device 10. Specifically, the processing circuit 110 uses the acquisition function 110f to acquire the output signals from the ultrasound diagnostic device 10, which are complex number data, after complex number delay and synthesis processing has been applied to the RF input signals for the number of channels. This output signal is then used as input data x, which is complex number data, and the complex number data after clutter removal is used as output data d to create the training dataset D.

[0108] As another example, the processing circuit 110 creates a training dataset D for a complex DNN for noise reduction filtering in B-mode signal processing, based on complex signal data acquired from the ultrasound diagnostic device 10. Specifically, the processing circuit 110 uses the acquisition function 110f to acquire the IQ signal in B-mode signal processing from the ultrasound diagnostic device 10 as input data x, which is complex number data, and the complex number data of the IQ signal after noise reduction as output data d to create the training dataset D.

[0109] As another example, the processing circuit 110 performs MTR filtering based on the complex signal data acquired from the ultrasound diagnostic device 10 to create a training dataset D for a complex DNN to remove the clutter component from the IQ signal in color Doppler mode. Specifically, the processing circuit 110 uses the acquisition function 110f to acquire the IQ signal containing the clutter component from the IQ signal in color Doppler mode from the ultrasound diagnostic device 10 as input data x, which is complex number data, and uses the complex number data of the IQ signal from which the clutter component has been removed as output data d to create the training dataset D.

[0110] As another example, the processing circuit 110 creates a training dataset D for a DNN (Deep Neural Network) to remove noise or as a segmentation tool, based on multidimensional data acquired from the ultrasound diagnostic device 10. Specifically, the processing circuit 110, using its acquisition function 110f, acquires power, velocity, and variance data obtained from the ultrasound diagnostic device 10 through autocorrelation processing in Doppler signal processing, as input data x, which is three-dimensional data. It then creates a training dataset D using the noise-removed three-dimensional data as output data d. As mentioned above, three-dimensional data can be associated with quaternion data, so the processing circuit 110 can acquire the three-dimensional data as quaternion data.

[0111] As another example, the processing circuit 110 creates a training dataset D for reducing noise and speckle in B-mode Doppler color images based on multidimensional data acquired from an image processing device related to ultrasound diagnosis. Specifically, the processing circuit 110 uses the acquisition function 110f to acquire multiple data from among B-mode signal, velocity, power, variance, etc., as multidimensional input data x, and creates a training dataset D using the noise-removed multidimensional data as output data d. As described above, 2D data can be associated with complex number data, and 3D or 4D data can be associated with quaternion data, so the processing circuit 110 can acquire the multidimensional data as complex number or quaternion data.

[0112] As another example, the processing circuit 110, using the acquisition function 110f, acquires multidimensional data related to ultrasound diagnosis from the magnetic resonance imaging apparatus 200 as input data x, and creates a training dataset D based on the output data d.

[0113] For example, the processing circuit 110 creates a training dataset D for a complex DNN to perform noise reduction filtering based on complex signal data acquired from the magnetic resonance imaging apparatus 200. Specifically, the processing circuit 110 uses the acquisition function 110f to acquire complex number data related to imaging for magnetic field disturbance estimation (ΔB0 estimation), phase contrast method, or SWI (Susceptibility-weighted imaging) method / QSM (Quantitative Susceptibility Mapping) method from the magnetic resonance imaging apparatus 200 as input data x, and creates a training dataset D using the denoised complex number data as output data d.

[0114] As another example, in the case of reconstruction by iterative optimization such as ADMM (Alternating Direction Method of Multipliers), the DNN may be incorporated as knowledge within the optimization operation, and the multidimensional numbers may be complex numbers, with the DNN being considered as part of the functions that constitute the optimization operation.

[0115] The processing circuit 110 may also output the information obtained by machine learning as a map.

[0116] As another example, the processing circuit 110 uses the acquisition function 110f to create a training dataset D for QRS complex detection and arrhythmia case classification from an electrocardiograph or the like. Specifically, the processing circuit 110 uses the acquisition function 110f to acquire vector cardiogram data from an electrocardiograph or the like as input data x, which is one-dimensional data of multidimensional data, and uses the data related to the detection results of the QRS complex and the arrhythmia case classification results as output data d to create the training dataset D.

[0117] As another example, the processing circuit 110 uses the acquisition function 110f to create a training dataset D for detecting organ information (e.g., position information and angle information) from various types of medical image processing devices. Specifically, the processing circuit 110 uses the acquisition function 110f to acquire medical data output as complex images from the medical image processing device as input data x, and uses data related to the organ detection results as output data d to create the training dataset D.

[0118] Returning to the flowchart in Figure 14, in step S100, the processing circuit 110, using the learning function 110b, sets the initial values ​​of the neural network's weight coefficients p=(p1, p2,…) p (0) Set the following: Here, p1, p2, ... are each weight coefficients between each node, and the weight coefficient p of the neural network is a single symbol representing the set of all those weight coefficients between each node.

[0119] Next, the processing circuit 110, using the learning function 110b, performs steps S110 to S130. Learning is performed through processing. Here, learning means finding the weight coefficients p that minimize the loss function E(p) for the given training dataset. Here, the loss function E(p) is calculated by taking the output of the neural network as y(x n Let p) be a suitable norm, and it can be given, for example, by equation (2) below.

[0120]

number

[0121] Here, the loss function E in the nth dataset n To minimize (p), the initial value of the neural network's weight coefficients p (0) Starting from this point, the weight coefficients p of the neural network after the tth iteration can be determined using methods such as stochastic gradient descent. (t) Based on this, the loss function E n The weight coefficients of the neural network after the t+1th iteration such that (p) becomes small. (t+1) We just need to calculate this. Here, let ε be an infinitesimal quantity, and the weight coefficients p of the neural network after t+1 iterations (t+1) p is the weight coefficient of the neural network after the tth iteration. (t) Using this, it is given by equations (3) and (4) below.

[0122]

number

[0123]

number

[0124] That is, the loss function E n The problem of minimizing the loss function E in a neural network is a problem in which the loss function E nThis can be reduced to the problem of finding the coefficient obtained by differentiating with respect to the weight coefficient p. This procedure can be carried out using backpropagation, as is well known.

[0125] Specifically, with layer 0 as the input layer and layer L as the output layer, the weight coefficient between the j-th node in layer l and the i-th node in layer 1-1 is w. ji (l) Therefore, this weight coefficient w ji (l) Loss function E n The derivative of is the input to the j-th node in the l-th layer. j (l) The output from the j-th node in the l-th layer is z j (l) Let f be the activation function, δ j (l) If we let be the coefficient determined by the left side of equation (6), then equations (5) to (8) below hold.

[0126]

number

[0127]

number

[0128]

number

[0129]

number

[0130] Here, as can be seen from equation (7), the coefficient δ relating to the (l+1)th term k (l+1) Based on this, the coefficient δ for the l term k (l)This allows us to calculate the derivative of the loss function with respect to the weight coefficients, and based on this, we can sequentially calculate the derivative with respect to the weight coefficients. This process is called backpropagation.

[0131] As described above, in step S110, the processing circuit 110 uses the learning function 110b to calculate the derivative of the loss function with respect to the weight coefficient using back error propagation, for example, using equations (5) to (8).

[0132] Next, in step S120, the processing circuit 110 updates the weight coefficients using the learning function 110b, for example, equation (3), to minimize the loss function.

[0133] Here, if the processing circuit 110 with the learning function 110b determines that the weight coefficient values ​​have converged (step S130 Yes), it terminates the machine learning process. On the other hand, if the processing circuit 110 with the learning function 110b determines that the weight coefficient values ​​have not converged (step S130 No), the process returns to step S110, and the weight coefficient values ​​are updated.

[0134] The embodiments are not limited to the examples described above. In the data in Figure 14, the multidimensional data used for training is described as being of one type; for example, if it is a complex number, all the data used for training is a complex number, and if it is a real number, all the data used for training is a real number. However, the embodiments are not limited to this, and for example, both complex number and real number data may be mixed and used for training.

[0135] Let's consider this case. As a first method, since real numbers are also complex numbers with an imaginary part of 0, real number data is considered as complex number data with an imaginary part of 0, and the processing circuit 110 having a learning function 110b learns about real number data and complex number data using a complex number neural network.

[0136] As a second method, the processing circuit 110 having a learning function 110b performs learning using a first neural network for real number data and a second neural network for complex number data. The first neural network is a real number neural network. The second neural network is a set of multiple neural networks.

[0137] At this point, in step S110 of Figure 14, the loss function is defined by the following equation (9).

[0138]

number

[0139] Here, E represents the loss function for the entire neural network, including the first and second neural networks. real The loss function for the first neural network is E complex The second neural network's loss function is shown. R(p) represents the regularization term, and λ1 and λ2 represent predetermined coefficients. x is the input, p is the weight coefficient, f is the activation function for the entire neural network, and f real F is the activation function in the first neural network. complex is the activation function in the second neural network, y is the output result of the entire neural network, y real is the output result in the first neural network, y complex This shows the output result from the second neural network.

[0140] Since the learning process is the same except for the expression of the loss function, the processing circuit 110 having the learning function 110b can perform the same learning process as explained in Figure 14, while still using equation (9) above as the loss function.

[0141] Furthermore, as a third method, the processing circuit 110 having the learning function 110b performs learning using a first neural network for real number data, and using the first neural network and a second neural network different from the first neural network for complex number data. That is, in some processes, the processing circuit 110 having the learning function 110b performs learning using a common neural network for both real number data and complex number data, and in some processes, it performs learning using a neural network specific to complex number data. For example, the processing circuit 110 having the learning function 110b performs learning using a common neural network for absolute value data and for real number data for polar coordinate data, and for argument data, it performs learning using a neural network specific to complex number data.

[0142] The embodiments are not limited to the examples described above. For example, learning may be performed using a neural network that simultaneously incorporates multiple representations of a multidimensional number. In other words, the neural network used by the processing circuit 100 for learning by the learning function 110b may be configured to include a plurality of subnetworks in which the multidimensional numbers are represented in different coordinate systems, and a subnetwork that connects at least two of these subnetworks.

[0143] First, let me briefly explain the background.

[0144] A multidimensional number can have multiple representations. A representation is obtained by expanding data using a defined basis and calculating, for example, the expansion coefficients. The basis being expanded and its expansion coefficients constitute the representation. For example, if a complex number is chosen as the multidimensional number, the representation of the complex number in the usual Cartesian coordinate system and its representation in polar coordinate system are examples of the multiple representations that a multidimensional number can have.

[0145] Here, the most efficient way to process the data—for example, using a coordinate system—varies depending on the nature of the data being trained. For example, with complex numbers, representing the data in a standard Cartesian coordinate system is often the most efficient. On the other hand, if there is rotational symmetry around the origin in complex space, representing the data in polar coordinates may be more efficient.

[0146] Therefore, for example, if it is desirable to perform learning in a Cartesian coordinate system, then it is advisable to perform learning in a Cartesian coordinate system, and if it is desirable to perform learning in a polar coordinate system, then it is advisable to perform learning in a polar coordinate system.

[0147] However, there are cases where it is difficult to determine from prior knowledge whether to perform learning in a Cartesian coordinate system or a polar coordinate system. In such situations, it is desirable for the processing circuit 110 to automatically determine whether to perform learning in a Cartesian coordinate system or a polar coordinate system. Furthermore, there are cases where it is desirable to perform learning in an "intermediate" coordinate system between a Cartesian coordinate system and a polar coordinate system, for example, a coordinate system that has 50% elements of a Cartesian coordinate system and 50% elements of a polar coordinate system. In such situations as well, it is desirable for the processing circuit 110 to perform learning appropriately.

[0148] Therefore, in this embodiment, first, multiple subnetworks, each representing a multidimensional in a different coordinate system, are combined to form a single composite network. For example, a subnetwork represented in a Cartesian coordinate system and a subnetwork represented in a polar coordinate system are combined to form a single composite network. With this configuration, the processing circuit 110 can automatically determine which coordinate system to use for learning and switch the coordinate system being learned.

[0149] For example, if it is desirable to perform learning in a Cartesian coordinate system, and learning is performed entirely in a Cartesian coordinate system, such a state can be represented in the composite network as a state where the weight coefficient of the subnetwork represented in the Cartesian coordinate system is 1, and the weight coefficient of the subnetwork represented in the polar coordinate system is 0. Similarly, if it is desirable to perform learning in a polar coordinate system, and learning is performed entirely in a polar coordinate system, such a state can be represented in the composite network as a state where the weight coefficient of the subnetwork represented in the Cartesian coordinate system is 0, and the weight coefficient of the subnetwork represented in the polar coordinate system is 1. Furthermore, for example, if it is desirable to perform learning in an "intermediate" coordinate system between the Cartesian and polar coordinate systems, for example, a coordinate system with 50% elements of the Cartesian coordinate system and 50% elements of the polar coordinate system, it can be represented as a state where the weight coefficient of the subnetwork represented in the Cartesian coordinate system is 0.5, and the weight coefficient of the subnetwork represented in the polar coordinate system is 0.5. In such cases, for example, the processing circuit 110 inputs data to both subnetworks, weights and adds their respective output results together to obtain the final output result. In this way, to enable simultaneous processing in multiple coordinate systems, subnetworks connecting multiple subnetworks, each representing a different coordinate system, are introduced as appropriate.

[0150] Regarding the learning method for the composite network, learning can be performed using, for example, the method already described. A difference from the previously described embodiment is that in this embodiment, in addition to the weight coefficients in each subnetwork, the weight coefficients representing the degree of connectivity between subnetworks are also subject to learning. For example, as explained in the above example, the coefficient representing the degree of connectivity between the Cartesian coordinate system subnetwork and the polar coordinate system subnetwork is also subject to learning. Note that the term "weight coefficients representing the degree of connectivity between subnetworks" is used here for convenience of explanation; in reality, such a single scalar quantity does not need to be explicitly implemented as a neural network. It is sufficient that such behavior is effectively represented in the overall weight coefficients of the neural network.

[0151] By adopting this configuration, the multidimensional arithmetic device according to the embodiment can automatically determine a suitable coordinate system according to the nature of the input data and process the input data in the optimal coordinate system. Alternatively, it can perform appropriate processing on input data that is desirable to be processed simultaneously and in parallel using multiple coordinate systems, or on data that is difficult to process appropriately using a single coordinate system, thereby enabling efficient machine learning.

[0152] Figures 15 to 17 will be used to explain this configuration in more detail.

[0153] As shown in Figure 15, in the multidimensional arithmetic device according to this embodiment, first, the input data 320 is converted into input data in which the multidimensional is expressed in different coordinate systems. Then, these input data are input to the neural network 300. For example, input data 321 in which the multidimensional related to input data 320 is expressed in a first coordinate system, input data 322 in which the multidimensional related to input data 320 is expressed in a second coordinate system, ..., input data 330 in which the multidimensional related to input data 320 is expressed in the nth coordinate system are input to the neural network 300.

[0154] The neural network 300 is composed of multiple subnetworks in which the multidimensionality is represented in different coordinate systems, and subnetworks that connect at least two of these subnetworks. For example, the neural network 300 is composed of a first subnetwork 301 in which the multidimensionality is represented in a first coordinate system, a second subnetwork 302 in which the multidimensionality is represented in a second coordinate system, ..., an nth subnetwork 310 in which the multidimensionality is represented in an nth coordinate system, and a subnetwork 350 that connects different subnetworks.

[0155] Furthermore, the neural network 300 outputs output data in which each multidimensional is represented in a different coordinate system. For example, output data 331 in which the multidimensional is represented in the first coordinate system, output data 332 in which the multidimensional is represented in the second coordinate system, ..., output data 340 in which the multidimensional is represented in the nth coordinate system are output from the neural network 300.

[0156] Next, the processing circuit 110 integrates the output data, each of which is represented in a different coordinate system, and outputs it as output data 341.

[0157] Next, using Figures 16 and 17, we will describe a more specific configuration example of the neural network 300 according to the embodiment. Figures 16 and 17 describe a case where complex numbers are used as the multidimensional numbers, and the multidimensional numbers are represented in two coordinate systems, a first coordinate system and a second coordinate system, where the first coordinate system is a Cartesian coordinate system and the second coordinate system is a polar coordinate system.

[0158] In such cases, as shown in Figure 16, input data 321 expressed in a Cartesian coordinate system is generated based on the complex numerical input data 320. Specifically, if one of the complex numbers z in the input data 320 is a complex number z = x + iy, with x as the real part and y as the imaginary part, then x and y become the input data 321 expressed in a Cartesian coordinate system. That is, for every 1 node in the input data 320, there are 2 nodes in the input data 321 expressed in a Cartesian coordinate system. If the number of nodes in the input data 320 is N, the number of nodes in the input data 321 expressed in a Cartesian coordinate system will be 2N, and the data related to these 2N nodes is input to the neural network 300.

[0159] Furthermore, if one of the complex numbers z in the input data 320 is z = r(cosθ + isinθ), then the radial vector r and the argument θ become the input data 322 expressed in polar coordinates. That is, for every node in the input data 320, there are 2 nodes in the input data 322 expressed in polar coordinates. If the number of nodes in the input data 320 is N, the number of nodes in the input data 322 expressed in polar coordinates will be 2N, and the processing circuit 110 inputs the data related to these 2N nodes into the neural network 300.

[0160] Furthermore, from the output data 331, which is expressed in Cartesian coordinates, one set of data x and y gives a complex number z = x + iy, which is input to the node of output data 341. In other words, for every two nodes of output data 331, one complex number data is input to the node of output data 341. Also, from the output data 332, which is expressed in polar coordinates, one set of data r and θ gives a complex number z = r(cosθ + isinθ), which is input to the node of output data 341. In other words, the processing circuit 110 inputs one complex number data for every two nodes of output data 332 to the node of output data 341.

[0161] Here, the processing circuit 110 accepts two inputs for each node corresponding to a single complex number data: data input based on output data 331 expressed in a Cartesian coordinate system, and data input from output data 331 expressed in a polar coordinate system. The processing circuit 110 generates complex number output data 341 for that node by simply averaging or weighting these two inputs.

[0162] Next, the configuration of the neural network 300 will be explained. As shown in Figure 16, the neural network 300 consists of multiple layers, for example. For example, the neural network 300 consists of multiple layers such as neural network 300a, neural network 300b, ..., neural network 300z. Each of these multiple layers is called, for example, an MR-NN (Multi-Representation Neural Network). The neural network 300 is constructed by connecting multiple MR-NNs in series, for example.

[0163] Figure 17 shows a more specific configuration of neural network 300a, one of the multiple layers that make up neural network 300. As shown in Figure 17, neural network 300a, which is an MR-NN and one of the multiple layers that make up neural network 300, accepts input data 321 expressed in a Cartesian coordinate system and input data 322 expressed in a polar coordinate system, and outputs output data 351 expressed in a Cartesian coordinate system and output data 352 expressed in a polar coordinate system. Output data 351 and output data 352 become input data for the next layer, neural network 300b. In other words, the output data of one layer of neural network 300 becomes the input data for the next layer.

[0164] Furthermore, the neural network 300a includes subnetworks 360, 361, 362, 363, and 364.

[0165] Here, subnetworks 360 and 362 are the first subnetworks in which complex numbers are represented in Cartesian coordinates, that is, multidimensional numbers are represented in the first coordinate system. Subnetwork 360 receives input from input data 321 represented in the first coordinate system, i.e., the Cartesian coordinate system, and outputs output data represented in the first coordinate system, i.e., the Cartesian coordinate system, to subnetworks 362 and 364.

[0166] Furthermore, subnetwork 362 receives input data expressed in a first coordinate system, i.e., a Cartesian coordinate system, from subnetworks 360 and 364, and outputs output data 351 expressed in the first coordinate system. For example, subnetwork 362 weights and adds the data input from subnetwork 360 and the data input from subnetwork 364.

[0167] Furthermore, subnetworks 361 and 363 are second subnetworks in which complex numbers are represented in polar coordinates, that is, multidimensional numbers are represented in a second coordinate system.

[0168] Here, subnetwork 361 receives input from input data 322 expressed in a second coordinate system, i.e., a polar coordinate system, and outputs output data expressed in the second coordinate system to subnetworks 363 and 364.

[0169] Furthermore, subnetwork 363 receives input data expressed in a second coordinate system from subnetworks 361 and 364, and outputs output data 352 expressed in a second coordinate system. For example, subnetwork 363 performs weighted addition of the data input from subnetwork 361 and the data input from subnetwork 364.

[0170] Furthermore, subnetwork 364 is a third subnetwork that connects the first subnetwork and the second subnetwork. Subnetwork 364 accepts input data expressed in the first coordinate system from subnetwork 360 and input data expressed in the second coordinate system from subnetwork 361. Subnetwork 364 also outputs output data expressed in the first coordinate system to subnetwork 362 and output output data expressed in the second coordinate system to subnetwork 363. As an example, subnetwork 364 takes the data input from subnetwork 360 and the data input from subnetwork 361, standardizes the data format, and then, for example, expresses both sets of data in the first coordinate system, adds them together with weights, outputs the resulting data expressed in the first coordinate system as is to subnetwork 362, and converts it to data expressed in the second coordinate system before outputting it to subnetwork 363.

[0171] Thus, each of the multiple layers constituting the neural network 300 includes a first subnetwork in which the multidimensional is represented in a first coordinate system, a second subnetwork in which the multidimensional is represented in a second coordinate system different from that of the first coordinate system, and a third subnetwork connecting the first subnetwork and the second subnetwork. To explain the correspondence between the first subnetwork 301, the second subnetwork 302, ..., and the subnetwork 350 connecting different subnetworks in Figure 15 and the elements in Figure 17, subnetworks 360 and 362 in each layer of the neural network constituting the neural network 300 are examples of the first subnetwork 301 in Figure 15, subnetworks 361 and 363 in each layer of the neural network constituting the neural network 300 are examples of the second subnetwork 302 in Figure 15, and subnetwork 364 in each layer of the neural network constituting the neural network 300 is an example of the subnetwork 350 connecting different subnetworks in Figure 15.

[0172] The processing circuit 110 uses the learning function 110b to train the neural network 300, for example, in the same manner as described in Figure 14. Processes already described are omitted here.

[0173] In creating training data, the processing circuit 110 creates the training dataset D using the training data creation function 110a. Here, the training dataset D is, for example, D = {{x 11 , x 12、 ..., x 1p} 、 {x 21 , x 22、 ..., x 2p}, ..., {x q1 , x q2、 ..., x qp}}. Here, p and q are natural numbers, and x jk This is multidimensional data x jThis represents data expressed in the k-th coordinate system. For example, the j-th multidimensional data x j If the data is a complex number, and the first and second coordinate systems are a Cartesian coordinate system and a polar coordinate system, then x j1 This is the j-th complex number data x represented in a Cartesian coordinate system. j and x j2 This is the j-th complex number data x expressed in polar coordinates. j That is the case.

[0174] Next, for these training datasets D, an appropriate loss function is determined, and the processing circuit 110 adjusts the values ​​of the weight coefficients of the neural network 300 using the learning function 110b so that the loss function is minimized. The loss function may be, for example, the sum of appropriate weights of the loss functions determined in each coordinate system subnetwork. Alternatively, for the loss function of the polar coordinate system subnetwork, for example, the product of the loss function related to the argument angle θ and the loss function related to the radius vector r may be selected as the loss function of the polar coordinate system subnetwork. The loss function may also include a regularization term.

[0175] According to at least one embodiment described above, machine learning can be performed efficiently.

[0176] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0177] With respect to the above embodiments, the following additional notes are disclosed as aspects of the invention and selective features.

[0178] (Note 1) An acquisition unit that acquires data including multidimensional numbers, An input section takes a parametric function as input, in which the function shape for the first element and the function shape for a second element (which is different from the first element) are different. An application unit that takes data containing multiple dimensions as input and applies it to a parametric function to output output data, A multidimensional arithmetic unit equipped with a multidimensional arithmetic unit.

[0179] (Note 2) A magnetic resonance imaging apparatus comprising a sequence control unit for executing pulse sequences and a multidimensional arithmetic unit, The acquisition unit acquires the data obtained by the pulse sequence as data including multiple dimensions. The application unit outputs magnetic resonance images as output data.

[0180] (Note 3) Data containing multiple dimensions may be k-space data, hybrid-space data, or image data. The parametric function may also be a denoising function. The output data may be k-space data, hybrid space data, or image data.

[0181] (Note 4) The parametric function may also be a map estimation function. The output data may be magnetic field disturbance data, flow velocity data, magnetic susceptibility data, or pseudo-magnetic susceptibility data.

[0182] (Note 5) An ultrasound diagnostic apparatus comprising an ultrasound probe and a multidimensional computing device, The acquisition unit acquires data obtained by the ultrasonic probe as data containing multiple dimensions. The application unit outputs ultrasound diagnostic images as output data.

[0183] (Note 6) Data containing multiple dimensions may be any of the following: ultrasound RF data, IQ data, Doppler data, or image data. The parametric function may be a noise reduction or speckle removal function. The output data may be any of the following: ultrasound RF data, IQ data, Doppler data, or image data.

[0184] (Note 7) Data containing multiple dimensions may be either ultrasound RF data or IQ data. The parametric function may also be an acoustic clutter rejection filter function in beamforming. The output data may be either ultrasound RF data or IQ data.

[0185] (Note 8) Data containing multiple dimensions may also be ultrasound IQ data. The parametric function may also be an MTR filter function. The output data may also be ultrasound IQ data.

[0186] (Note 9) An acquisition unit that acquires data including multidimensional numbers, An input section takes a parametric function as input, in which the function shape for the first element and the function shape for a second element (which is different from the first element) are different. An application unit that takes data containing multiple dimensions as input and applies it to a parametric function to output output data, A multidimensional arithmetic unit equipped with a multidimensional arithmetic unit.

[0187] (Note 10) Data containing multiple dimensions may also be medical data. The parametric function may also be an object detection function. The output data may also represent the presence or absence of the object.

[0188] (Note 11) The input unit uses a first neural network for real number data and a second neural network for complex number data to perform learning. The first neural network is a real-number neural network. The second type of neural network is a set of multiple neural networks.

[0189] (Note 12) The input unit performs learning by using the sum of the loss function for the first neural network and the loss function for the second neural network as the loss function.

[0190] (Note 13) The input unit uses a first neural network for real number data, and a second neural network (different from the first) for complex number data to perform learning.

[0191] (Note 14) The input unit uses a common neural network for learning both real and complex number data in some processes, and a neural network specific to complex number data in other processes.

[0192] (Note 15) The input unit uses a common neural network for learning absolute value data from polar coordinate data, and a data-specific neural network for learning complex number data related to argument. [Explanation of Symbols]

[0193] 100 Data Processing Devices 110 Processing Circuit 110a Training data creation function 110b Learning function 110c interface function 110d Control Function 110e Applicable Functions 110f acquisition function

Claims

1. A data acquisition unit that acquires data including complex numbers, An input unit that acquires a trained neural network by inputting training data into a neural network, which consists of a first training input data expressed in a Cartesian coordinate system, a second training input data expressed in a polar coordinate system with the same data values, and ground truth data paired with the first and second training input data. An application unit receives data including a first inference input data in which the values ​​of the inference input data, which include complex numbers, are expressed in a Cartesian coordinate system, and a second inference input data in which the same values ​​of the inference input data are expressed in a polar coordinate system, and applies this data to the trained neural network to output output data that includes complex numbers expressed in a Cartesian coordinate system or a polar coordinate system. A multidimensional arithmetic unit equipped with a multidimensional arithmetic unit.

2. The multidimensional arithmetic device according to claim 1, wherein the neural network includes a parametric function relating to the argument in the pole coordinate representation of a complex number.

3. A medical image diagnostic apparatus comprising a scanning unit and a multidimensional arithmetic device as described in claim 1, wherein the acquisition unit acquires the data obtained by the scanning unit as data including the complex number.

4. The scanning unit is an ultrasonic probe, The medical image diagnostic apparatus according to claim 3, wherein the application unit outputs an ultrasound diagnostic image as output data.

5. The scanning unit executes a pulse sequence to collect magnetic resonance signals. The medical image diagnostic apparatus according to claim 3, wherein the application unit outputs a magnetic resonance image as output data.

Images