Machine learning model, program, ultrasound diagnostic device, ultrasound diagnostic system, image processing device and training device

Abstract

To provide a technology to efficiently generate time change image data used for training of a machine learning model.SOLUTION: The present invention is related to a machine learning model trained using training data including a pair of at least one training time change image data of second time change image data obtained by standardizing, in a time direction, first time change image data based on a reception signal for image generation received by an ultrasonic probe, and third time change image data based on the second time change image data, and correct answer data including a detection object corresponding to the at least one training time change image data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a machine learning model, a program, an ultrasound diagnostic device, an ultrasound diagnostic system, an image processing device, and a training device. [Background technology]

[0002] With the recent advances in deep learning technology, machine learning models are now being used for a variety of purposes. For example, in the medical field, attempts are being made to use machine learning in imaging diagnostics such as ultrasound diagnosis.

[0003] A typical ultrasound diagnostic device can operate in two modes: Doppler mode and M mode, in addition to B mode, which displays cross-sectional images, and color Doppler mode, which displays blood flow images. Doppler mode is an imaging mode that displays the spectrum of changes in blood flow (tissue) velocity, and includes pulse wave Doppler (PWD), continuous wave Doppler (CWD), and tissue Doppler imaging (TDI). On the other hand, M mode is a mode that displays the time change of a single line on a cross-sectional image (B mode). Various imaging diagnostic techniques using ultrasound images have been proposed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-181058 [Patent Document 2] Japanese Patent Application Laid-Open No. 2006-197969 [Patent Document 3] Patent Publication No. 2021-164533 Summary of the Invention [Problem to be solved by the invention]

[0005] In Doppler mode, if the sweep speed of the ultrasound diagnostic device is different, the width of the spectrum in the time direction in the ultrasound image may vary even for the same patient. If the sweep speed is low, the spectrum in the ultrasound image will shrink in the time direction, and if the sweep speed is high, the spectrum will expand in the time direction. In addition, heart rates (HR) may vary depending on the patient. If the heart rate is fast, the spectrum in the ultrasound image will be congested in the time direction, and if the heart rate is slow, the spectrum will expand in the time direction.

[0006] In M-mode, if the sweep speed is slow, the M-mode image shrinks in the time direction, and if the sweep speed is fast, the M-mode image stretches in the time direction. Also, if the heart rate is fast, the M-mode image becomes congested in the time direction, and if the heart rate is slow, the M-mode image expands in the time direction.

[0007] In time-varying image modes such as Doppler mode and M mode, the time-varying image data changes as if it is being stretched or compressed in the time direction depending on the settings such as the sweep speed. Therefore, when an AI (Artificial Intelligence) model is applied to automatic measurement of time-varying images, the number of patterns required to train the AI ​​model increases depending on the sweep speed. For this reason, learning data (training data) of time-varying images must be prepared for each sweep speed, and if there is a small amount of training data, the performance of the AI ​​processing can be reduced.

[0008] Furthermore, in the time-varying imaging mode, even if the sweep speed is the same, if the subject's heart rate is different, the acquired data may appear to expand or contract in the time direction. Therefore, differences in heart rate can cause the same problems as those caused by differences in sweep speed.

[0009] In view of the above problems, one object of the present disclosure is to provide a technology for efficiently generating time-varying image data used to train machine learning models. [Means for solving the problem]

[0010] According to one aspect of the present disclosure, there is provided a method for generating an image by using an ultrasound probe, the method comprising: generating a first time-varying image data based on a received signal for generating an image, the first time-varying image data being normalized in a time direction; Further data expansion was performed to transform the data in the time direction. a machine learning model trained using training data including pairs of at least one training time-varying image data and a third time-varying image data, and ground truth data including a detection target corresponding to the at least one training time-varying image data, 1st The normalization of the time-varying image data in the time direction is 1st The image of the time-varying image data whose width direction is the time axis direction is enlarged and / or reduced. 1st Sweep speed when generating time-varying image data to Based on the above 1st The time-varying image data is processed by a computer based on the received signals for generating an image received by an ultrasonic probe. The second prediction target time-varying image data is normalized in the time direction. This relates to a trained machine learning model that functions to output a detection target as an inference result from first prediction target time-varying image data. [Effects of the Invention]

[0011] According to the present disclosure, it is possible to efficiently generate time-varying image data that can be used to train machine learning models. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram illustrating normalization of training data according to one embodiment of the present disclosure. [Figure 2] FIG. 2 is a schematic diagram illustrating normalization of training data according to one embodiment of the present disclosure. [Figure 3] FIG. 3 is a schematic diagram illustrating a training process for a machine learning model according to one embodiment of the present disclosure. [Figure 4] FIG. 4 is a schematic diagram illustrating an inference process of a machine learning model according to one embodiment of the present disclosure. [Figure 5]FIG. 5 is a schematic diagram illustrating an ultrasound diagnostic process according to one embodiment of the present disclosure. [Figure 6] FIG. 6 is a schematic diagram illustrating an ultrasound diagnostic process according to one embodiment of the present disclosure. [Figure 7] FIG. 7 is a schematic diagram illustrating an ultrasound diagnostic process according to one embodiment of the present disclosure. [Figure 8] FIG. 8 is a diagram showing the appearance of an ultrasound diagnostic apparatus according to an embodiment of the present disclosure. [Figure 9] FIG. 9 is a block diagram showing a hardware configuration of an ultrasound diagnostic apparatus according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a block diagram illustrating the hardware configuration of a training device and an image processing device according to an embodiment of the present disclosure. [Figure 11] FIG. 11 is a block diagram illustrating a functional configuration of a training device according to an embodiment of the present disclosure. [Figure 12] FIG. 12 is a schematic diagram illustrating normalization of time-varying image data according to one embodiment of the present disclosure. [Figure 13] FIG. 13 is a schematic diagram illustrating normalization of time-varying image data according to one embodiment of the present disclosure. [Figure 14] FIG. 14 is a schematic diagram illustrating normalization of time-varying image data according to one embodiment of the present disclosure. [Figure 15] FIG. 15 is a block diagram showing the functional configuration of an ultrasound diagnostic apparatus according to an embodiment of the present disclosure. [Figure 16] FIG. 16 is a diagram illustrating a VTI measurement process according to an embodiment of the present disclosure. [Figure 17] FIG. 17 is a diagram illustrating a VTI measurement process according to an embodiment of the present disclosure. [Figure 18] FIG. 18 is a diagram illustrating a training process for VTI measurement according to one embodiment of the present disclosure. [Figure 19] FIG. 19 is a diagram illustrating an inference process for VTI measurement according to one embodiment of the present disclosure. [Figure 20] FIG. 20 is a flowchart illustrating a VTI measurement process according to an embodiment of the present disclosure. [Figure 21] FIG. 21 is a diagram illustrating a VTI measurement process according to an embodiment of the present disclosure. [Figure 22] FIG. 22 is a diagram illustrating a training process for VTI measurement according to one embodiment of the present disclosure. [Figure 23] FIG. 23 is a diagram illustrating an inference process for VTI measurement according to one embodiment of the present disclosure. [Figure 24] FIG. 24 is a flowchart illustrating a VTI measurement process according to an embodiment of the present disclosure. [Figure 25] FIG. 25 is a diagram illustrating a PSV·EDV measurement process according to an embodiment of the present disclosure. [Figure 26] FIG. 26 is a diagram illustrating a training process for measuring PSV and EDV according to one embodiment of the present disclosure. [Figure 27] FIG. 27 is a diagram illustrating an inference process for measuring PSV and EDV according to one embodiment of the present disclosure. [Figure 28] FIG. 28 is a flowchart illustrating a PSV·EDV measurement process according to an embodiment of the present disclosure. [Figure 29] FIG. 29 is a diagram illustrating a PSV·EDV measurement process according to an embodiment of the present disclosure. [Figure 30] FIG. 30 is a diagram illustrating a training process for measuring PSV and EDV according to one embodiment of the present disclosure. [Figure 31] FIG. 31 is a diagram illustrating an inference process for measuring PSV and EDV according to one embodiment of the present disclosure. [Figure 32] FIG. 32 is a flowchart illustrating a PSV·EDV measurement process according to an embodiment of the present disclosure. [Figure 33] FIG. 33 is a diagram illustrating a blood flow measurement process according to an embodiment of the present disclosure. [Figure 34] FIG. 34 is a diagram illustrating a blood flow measurement process according to an embodiment of the present disclosure. [Figure 35] FIG. 35 is a flowchart showing a blood flow measurement process according to an embodiment of the present disclosure. [Figure 36] FIG. 36 is a diagram illustrating an E / A ratio measurement process according to an embodiment of the present disclosure. [Figure 37] FIG. 37 is a diagram illustrating an E / A ratio measurement process according to an embodiment of the present disclosure. [Figure 38] FIG. 38 is a flowchart illustrating an E / A ratio measurement process according to an embodiment of the present disclosure. [Figure 39] FIG. 39 is a diagram illustrating an E / A ratio measurement process according to an embodiment of the present disclosure. [Figure 40] FIG. 40 is a flowchart illustrating an E / A ratio measurement process according to an embodiment of the present disclosure. [Figure 41] FIG. 41 is a diagram illustrating a TAPSE measurement process according to an embodiment of the present disclosure. [Figure 42] FIG. 42 is a diagram illustrating a TAPSE measurement process according to an embodiment of the present disclosure. [Figure 43] FIG. 43 is a diagram illustrating a training process for TAPSE measurement according to one embodiment of the present disclosure. [Figure 44] FIG. 44 is a diagram illustrating an inference process for TAPSE measurement according to one embodiment of the present disclosure. [Figure 45] FIG. 45 is a flowchart illustrating a TAPSE measurement process according to an embodiment of the present disclosure. [Figure 46] FIG. 46 is a diagram illustrating a TAPSE measurement process according to one embodiment of the present disclosure. [Figure 47] FIG. 47 is a diagram illustrating a training process for TAPSE measurement according to one embodiment of the present disclosure. [Figure 48] FIG. 48 is a diagram illustrating an inference process for TAPSE measurement according to one embodiment of the present disclosure. [Figure 49] FIG. 49 is a diagram illustrating a training process for TAPSE measurement according to one embodiment of the present disclosure. [Figure 50] FIG. 50 is a diagram illustrating an inference process for TAPSE measurement according to one embodiment of the present disclosure. [Figure 51] FIG. 51 is a flowchart illustrating a TAPSE measurement process according to an embodiment of the present disclosure. [Figure 52] FIG. 52 is a diagram illustrating an IVC diameter measurement process according to one embodiment of the present disclosure. [Figure 53] FIG. 53 is a diagram illustrating a training process for IVC diameter measurement according to one embodiment of the present disclosure. [Figure 54] FIG. 54 is a diagram illustrating an inference process for IVC diameter measurement according to one embodiment of the present disclosure. [Figure 55] FIG. 55 is a flowchart illustrating an IVC diameter measurement process according to an embodiment of the present disclosure. [Figure 56] FIG. 56 is a diagram illustrating an IVC diameter measurement process according to one embodiment of the present disclosure. [Figure 57] FIG. 57 is a flowchart illustrating an IVC diameter measurement process according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0014] In the following embodiment, an image processing device that uses a machine learning model to output a detection target from a time-varying image acquired from a subject is disclosed.

[0015] [overview] It is generally known that time-varying images acquired by an ultrasound diagnostic device at different sweep speeds for the same subject differ. For example, Figure 1 shows time-varying images acquired at three sweep speeds: 30 mm / sec, 40 mm / sec, and 60 mm / sec. The three time-varying images reveal different spectral widths in the time direction. That is, at slower sweep speeds, the spectrum on the time-varying image shrinks in the time direction, whereas at faster sweep speeds, the spectrum expands in the time direction. That is, the spectrum of the time-varying image acquired at 30 mm / sec is more compressed in the time direction than the spectrum of the time-varying image acquired at 60 mm / sec. Thus, in time-varying image modes, such as Doppler mode and M-mode, of an ultrasound diagnostic device, the time-varying image acquired at the sweep speed set by the user may expand or contract in the time direction. Therefore, when applying a machine learning model to time-varying image data, the number of patterns required for training the machine learning model increases depending on the sweep speed. Therefore, to achieve good model prediction performance, it is necessary to prepare training data for various sweep speeds.

[0016] In the following examples, data extension is performed on time-varying image data acquired by an imaging diagnostic device, and the time-varying image data is normalized to a predetermined sweep speed. The normalized time-varying image data is used to perform training and inference processes for a machine learning model. For example, as shown in FIG. 1 , two sets of training time-varying image data, 30 mm / sec and 60 mm / sec, are normalized to a sweep speed of 40 mm / sec. Specifically, when time-varying image data with a sweep speed SS1 is normalized to a predetermined sweep speed SS0, the time-varying image with a sweep speed SS1 is enlarged or reduced by a factor of SS0 / SS1 in the time axis direction, i.e., the width direction of the image. That is, as shown in FIG. 1 , the time-varying image acquired with a sweep speed of 30 mm / sec is enlarged by a factor of 40 / 30 = 1.33, and the time-varying image acquired with a sweep speed of 60 mm / sec is reduced by a factor of 40 / 60 = 0.66. Time-varying images acquired with different sweep speeds for the same subject can be aligned by such normalization.

[0017] The enlarged and reduced time-varying images are then adjusted to fit the width of a predetermined image. Specifically, as shown in FIG. 1, the enlarged time-varying image may be cropped to fit the width of the time-varying image with a sweep speed of 40 mm / sec. On the other hand, the reduced time-varying image may be pasted onto a background image with the width of the time-varying image with a sweep speed of 40 mm / sec.

[0018] In this way, it is possible to acquire training data for a machine learning model using time-varying image data at a predetermined sweep rate that has been normalized from time-varying image data acquired at different sweep rates. This eliminates the need to prepare time-varying image data at each sweep rate as training data, making it possible to train a machine learning model using time-varying image data with a smaller number of samples. When using a machine learning model trained with normalized time-varying image data for inference processing, the time-varying image data to be inferred is preprocessed to be enlarged or reduced to time-varying image data at a predetermined sweep rate, and then input to the trained machine learning model.

[0019] Furthermore, for time-varying images acquired for different subjects and / or different heart rates, the time-varying image data may first be normalized to a predetermined sweep speed. As shown in FIG. 2, the time-varying image data may be normalized to a sweep speed of 40 mm / sec. Next, the time-varying image data normalized for sweep speed may be normalized to a predetermined heart rate. Here, normalization for heart rate may be performed by expanding or contracting the time-varying image in the time axis direction, i.e., the width direction of the image, similar to the normalization for sweep speed described above. In the example shown in FIG. 2, the time-varying image data may be normalized to a heart rate of 80 bpm.

[0020] The enlarged and reduced time-change image is then adjusted to fit the width of the predetermined image. Specifically, as shown in Fig. 2, the enlarged time-change image may be cropped to fit the width of the predetermined time-change image. On the other hand, the reduced time-change image may be pasted onto a background image having the same width as the predetermined time-change image.

[0021] In this way, it is possible to obtain training data for a machine learning model using normalized time-varying image data from time-varying image data acquired at different sweep speeds and heart rates. This eliminates the need to prepare time-varying image data for each sweep speed and each heart rate as training data, making it possible to train a machine learning model using a smaller number of time-varying image data samples. When using a machine learning model trained with normalized time-varying image data for inference processing, the time-varying image data to be inferred is preprocessed to enlarge or reduce the time-varying image data for a predetermined sweep speed and / or heart rate, and then input to the trained machine learning model.

[0022] For example, when training a machine learning model using time-varying image data generated by the above-described data augmentation, as shown in FIG. 3 , time-varying image data of different sweep speeds and / or heart rates are stored in a training data database (DB) 20 in association with ground truth data, and a training device 50 performs the above-described data augmentation on the time-varying image data obtained from the training data DB 20 to obtain standardized time-varying image data. The training device 50 trains a machine learning model 10 to be trained using the standardized time-varying image data. For example, if the machine learning model 10 to be trained is realized as any type of neural network such as a convolutional neural network, the training device 50 inputs the standardized time-varying image data into the neural network to be trained, and updates the parameters of the neural network to be trained according to any training algorithm such as backpropagation based on the error between the output result from the neural network and the ground truth data.

[0023] Once the training of the machine learning model 10 is completed in this manner, the trained machine learning model 10 receives normalized time-varying image data from the time-varying image to be predicted, and outputs an inference result, as shown in FIG. 4. For example, in ultrasound diagnosis, the inference result may be a specific position, timing, region, etc. represented by a spectrum on the time-varying image, as will be described in more detail below. As shown in FIG. 5, the ultrasound diagnostic device 100 transmits an ultrasound signal to the subject 30 and acquires time-varying image data based on a reflected signal from the subject 30. The ultrasound diagnostic device 100 normalizes the acquired time-varying image data, inputs the normalized time-varying image data to the trained machine learning model 10 provided by the training device 50, and outputs a processing result obtained by performing the reverse process of the preprocessing on the inference result acquired from the trained machine learning model 10.

[0024] 5, the ultrasound diagnostic device 100 stores the trained machine learning model 10 provided by the training device 50. However, the ultrasound diagnostic device 100 according to the present disclosure is not necessarily limited to this, and may utilize a trained machine learning model 10 stored in an external model database (DB) 40, such as on the cloud, as shown in FIG. 6. Specifically, the ultrasound diagnostic system 1 includes the ultrasound diagnostic device 100 and a model DB 40. The ultrasound diagnostic device 100 normalizes time-varying image data of the prediction target acquired from the subject 30 and passes the normalized time-varying image data to the model DB 40. Then, upon acquiring an inference result for the normalized time-varying image data from the model DB 40, the ultrasound diagnostic device 100 may output the acquired inference result.

[0025] Alternatively, as shown in FIG. 7 , an ultrasound diagnostic device 100 according to the present disclosure may communicate with an external image processing device 200, such as on the cloud. For example, the ultrasound diagnostic device 100 transmits received signals acquired in response to ultrasound signals transmitted to a subject 30 to the image processing device 200. The image processing device 200 normalizes the received signals and acquires inference results from the normalized received signals using a trained machine learning model 10. The image processing device 200 may store the acquired inference results and / or provide the inference results to the ultrasound diagnostic device 100.

[0026] [Hardware configuration of ultrasound diagnostic equipment] 8 is a diagram showing an example of the appearance of the ultrasonic diagnostic device 100. FIG. 9 is a block diagram showing an example of the configuration of the main parts of the control system of the ultrasonic diagnostic device 100.

[0027] The ultrasound diagnostic device 100 visualizes, as an ultrasound image, the shape or dynamics inside the subject 30. The ultrasound diagnostic device 100 according to this embodiment is used, for example, to capture an ultrasound image (i.e., a tomographic image) of a detection target region and to perform an examination on the detection target region.

[0028] 8, the ultrasonic diagnostic device 100 includes an ultrasonic diagnostic device main body 1010 and an ultrasonic probe 1020. The ultrasonic diagnostic device main body 1010 and the ultrasonic probe 1020 are connected via a cable 1030.

[0029] The ultrasonic probe (ultrasonic probe) 1020 transmits an ultrasonic beam (e.g., about 1 to 30 MHz) into the subject 30 (e.g., a human body), and also functions as an acoustic sensor that receives ultrasonic echoes of the transmitted ultrasonic beam reflected within the subject 30 and converts them into electrical signals.

[0030] The user performs an examination by bringing the transmitting / receiving surface of the ultrasonic beam of the ultrasonic probe 1020 into contact with the body surface of the target area of ​​the subject 30 and operating the ultrasonic diagnostic device 100. The ultrasonic probe 1020 may be any of a convex probe, a linear probe, a sector probe, a three-dimensional probe, or the like.

[0031] The ultrasonic probe 1020 includes, for example, a plurality of transducers (e.g., piezoelectric elements) arranged in a matrix, and a channel switching device (e.g., a multiplexer) for switching and controlling the driving state of the plurality of transducers on and off individually or in block units (hereinafter referred to as "channels").

[0032] Each transducer of the ultrasonic probe 1020 converts a voltage pulse generated in the ultrasonic diagnostic device main body 1010 (transmitting unit 1012) into an ultrasonic beam and transmits it into the subject 30, receives ultrasonic echoes reflected within the subject 30, converts them into electrical signals (hereinafter referred to as "received signals"), and outputs them to the ultrasonic diagnostic device main body 1010 (receiving unit 1013).

[0033] As shown in FIG. 9, the ultrasound diagnostic device main body 1010 includes an operation input unit 1011, a transmission unit 1012, a reception unit 1013, an ultrasound image generation unit 1014, a display image generation unit 1015, an output unit 1016, and a control unit 1017.

[0034] The transmitting unit 1012, receiving unit 1013, ultrasound image generating unit 1014, and display image generating unit 1015 are composed of dedicated or general-purpose hardware (electronic circuits) corresponding to each process, such as a DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), or PLD (Programmable Logic Device), and work in cooperation with the control unit 1017 to realize each function.

[0035] The operation input unit 1011 receives, for example, a command to start a diagnosis or input of information about the subject 30. The operation input unit 1011 may include, for example, an operation panel having a plurality of input switches, a keyboard, a mouse, etc. The operation input unit 1011 may be configured as a touch panel provided integrally with the output unit 1016.

[0036] The transmission unit 1012 is a transmitter that transmits a voltage pulse as a drive signal to the ultrasonic probe 1020 in accordance with instructions from the control unit 1017. The transmission unit 1012 may be configured to include, for example, a high-frequency pulse oscillator and a pulse setting unit. The transmission unit 1012 may adjust the voltage pulse generated by the high-frequency pulse oscillator to the voltage amplitude, pulse width, and transmission timing set by the pulse setting unit, and transmit the adjusted voltage pulse for each channel of the ultrasonic probe 1020.

[0037] The transmitting unit 1012 has a pulse setting unit for each of the multiple channels of the ultrasound probe 1020, and is capable of setting the voltage amplitude, pulse width, and transmission timing of the voltage pulse for each of the multiple channels. For example, the transmitting unit 1012 may change the target depth or generate different pulse waveforms by setting appropriate delay times for the multiple channels.

[0038] The receiving unit 1013 is a receiver that receives and processes received signals related to ultrasonic echoes generated by the ultrasonic probe 1020 in accordance with instructions from the control unit 1017. The receiving unit 1013 may be configured to include a preamplifier, an AD conversion unit, and a receive beamformer.

[0039] The receiver 1013 amplifies the received signals related to weak ultrasonic echoes for each channel using a preamplifier, and converts the received signals into digital signals using an AD converter.The receiver 1013 then combines the received signals of multiple channels into one signal using a receive beamformer by phasing and adding the received signals of each channel, thereby generating acoustic line data.

[0040] The ultrasound image generating unit 1014 acquires the received signal (acoustic line data) from the receiving unit 1013 and generates an ultrasound image (that is, a tomographic image) of the inside of the subject 30.

[0041] For example, when the ultrasonic probe 1020 transmits a pulsed ultrasonic beam in the depth direction, the ultrasonic image generation unit 1014 accumulates the signal intensities of the ultrasonic echoes detected thereafter in a line memory in a time-series manner. Then, as the ultrasonic beam from the ultrasonic probe 1020 scans inside the subject 30, the ultrasonic image generation unit 1014 sequentially accumulates the signal intensities of the ultrasonic echoes at each scanning position in the line memory to generate two-dimensional data in units of frames. Then, the ultrasonic image generation unit 1014 can generate an ultrasonic image representing a two-dimensional structure in a cross section including the ultrasonic transmission direction and the ultrasonic scanning direction by converting the signal intensities of the two-dimensional data into brightness values.

[0042] The ultrasound image generating unit 1014 may include, for example, an envelope detection circuit that performs envelope detection on the received signal acquired from the receiving unit 1013, a logarithmic compression circuit that performs logarithmic compression on the signal strength of the received signal detected by the envelope detection circuit, and a dynamic filter that is a bandpass filter whose frequency characteristics change according to the depth and removes noise components contained in the received signal.

[0043] The display image generation unit 1015 acquires ultrasound image data from the ultrasound image generation unit 1014 and generates a display image including a display area for the ultrasound image. Then, the display image generation unit 1015 sends the generated display image data to the output unit 1016. The display image generation unit 1015 may sequentially update the display image every time a new ultrasound image is acquired from the ultrasound image generation unit 1014, and cause the output unit 1016 to display the display image in a moving image format.

[0044] Furthermore, the display image generating unit 1015 may generate a display image in which, in accordance with an instruction from the control unit 1017, an image in which time-series data of the detection target is graphically displayed together with an ultrasound image is embedded in the display area.

[0045] The display image generating unit 1015 may generate a display image after performing predetermined image processing such as coordinate conversion processing and data interpolation processing on the ultrasound image output from the ultrasound image generating unit 1014.

[0046] The output unit 1016 acquires display image data from the display image generation unit 1015 and outputs the display image in accordance with instructions from the control unit 1017. For example, the output unit 1016 may be configured with a liquid crystal display, an organic EL display, a CRT display, or the like, and may display the display image.

[0047] The control unit 1017 controls the operation input unit 1011, the transmission unit 1012, the reception unit 1013, the ultrasound image generation unit 1014, the display image generation unit 1015, and the output unit 1016 according to their respective functions, thereby performing overall control of the ultrasound diagnostic apparatus 100.

[0048] The control unit 1017 may have a CPU (Central Processing Unit) 1171 as an arithmetic / control device, and a ROM (Read Only Memory) 1172 and RAM (Random Access Memory) 1173 as main storage devices. Basic programs and basic setting data are stored in the ROM 1172. The CPU 1171 reads out a program corresponding to the processing content from the ROM 172, stores it in the RAM 1173, and executes the stored program, thereby centrally controlling the operations of the functional blocks (transmission unit 1012, reception unit 1013, ultrasound image generation unit 1014, display image generation unit 1015, and output unit 1016) of the ultrasound diagnostic apparatus main body 1010.

[0049] [Hardware configuration of training device and image processing device] Next, the hardware configuration of the training device 50 and the image processing device 200 according to an embodiment of the present disclosure will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the hardware configuration of the training device 50 and the image processing device 200 according to an embodiment of the present disclosure.

[0050] The training device 50 and the image processing device 200 may each be realized by a computing device such as a server, a personal computer (PC), a smartphone, or a tablet, and may have, for example, a hardware configuration as shown in Fig. 10. That is, the training device 50 and the image processing device 200 each have a drive device 101, a storage device 102, a memory device 103, a processor 104, a user interface (UI) device 105, and a communication device 106, which are interconnected via a bus B.

[0051] The programs or instructions that realize the various functions and processes described below in the training device 50 and the image processing device 200 may be stored on a removable storage medium such as a CD-ROM (Compact Disk-Read Only Memory) or flash memory. When the storage medium is set in the drive device 101, the programs or instructions are installed from the storage medium to the storage device 102 or memory device 103 via the drive device 101. However, the programs or instructions do not necessarily have to be installed from the storage medium, and may be downloaded from any external device via a network or the like.

[0052] The storage device 102 is realized by a hard disk drive or the like, and stores installed programs or instructions as well as files, data, etc. used to execute the programs or instructions.

[0053] The memory device 103 is realized by a random access memory, a static memory, or the like, and when a program or instruction is activated, reads and stores the program, instruction, data, or the like from the storage device 102. The storage device 102, the memory device 103, and the removable storage medium may be collectively referred to as a non-transitory storage medium.

[0054] The processor 104 may be realized by one or more central processing units (CPUs), graphics processing units (GPUs), processing circuitry, etc., which may be composed of one or more processor cores, and performs various functions and processes of the model generation device 100 described below in accordance with programs, instructions, data such as parameters required to execute the programs or instructions, etc. stored in the memory device 103.

[0055] The user interface (UI) device 105 may be composed of input devices such as a keyboard, mouse, camera, microphone, etc., output devices such as a display, speaker, headset, printer, etc., and input / output devices such as a touch panel, and realizes an interface between the user and the training device 50 and image processing device 200. For example, the user operates the training device 50 and image processing device 200 by operating a GUI (Graphical User Interface) displayed on a display or touch panel using a keyboard, mouse, etc.

[0056] The communication device 106 is realized by various communication circuits that execute wired and / or wireless communication processing with external devices, the Internet, a LAN (Local Area Network), a cellular network, or other communication networks.

[0057] However, the above-described hardware configuration is merely an example, and the training device 50 and the image processing device 200 according to the present disclosure may be realized by any other appropriate hardware configuration.

[0058] [Normalization processing] Next, a normalization process for training and inference processes of a machine learning model used in ultrasound diagnosis according to one embodiment of the present disclosure will be described. Time-varying image data for training and inference may be acquired from the subject 30 at various sweep speeds and / or different heart rates. If the machine learning model were trained using time-varying image data at different sweep speeds and / or different heart rates, it would be necessary to prepare as many training time-varying image data sets required to train the machine learning model 10 for each sweep speed and / or heart rate, resulting in high costs for collecting the training data. According to the normalization process of this embodiment, the time-varying image data at different sweep speeds and / or different heart rates is normalized to a predetermined sweep speed and / or a predetermined heart rate, and the machine learning model 10 is trained using the normalized time-varying image data. This eliminates the need to collect as many training time-varying image data sets required for each sweep speed and / or heart rate to train the machine learning model 10, making it possible to generate the machine learning model 10 at lower cost.

[0059] 11 is a block diagram showing the functional configuration of a training device 50 according to one embodiment of the present disclosure. As shown in FIG. 11, the training device 50 according to this embodiment includes a pre-processing unit 51 and a training unit 52. For example, one or more functional units of the pre-processing unit 51 and the training unit 52 may be realized by one or more processors 104 executing one or more programs or instructions stored in the memory device 103.

[0060] The preprocessing unit 51 normalizes the time-varying image data. Specifically, the preprocessing unit 51 may acquire time-varying image data obtained by normalizing the time-varying image data in the time direction from the time-varying image data based on the received signals for image generation received by the ultrasound probe 1020. For example, the preprocessing unit 51 may convert the time-varying image data based on the signals received by the ultrasound probe 1020 from the subject 30 at a certain sweep speed into time-varying image data at a predetermined sweep speed. Specifically, to convert the training time-varying image data with a sweep speed of 30 mm / sec into time-varying image data with a predetermined sweep speed of 40 mm / sec, the preprocessing unit 51 may magnify the training time-varying image data with a sweep speed of 30 mm / sec by 40 / 30=1.33 times in the time direction, as shown in FIG. 12 .

[0061] Furthermore, the pre-processing unit 51 may acquire time-varying image data by further normalizing the enlarged time-varying image data. That is, to convert the time-varying image data with a heart rate of 72 bpm normalized with respect to the sweep speed into the predetermined heart rate of 80 bpm, the pre-processing unit 51 may reduce the time-varying image data with a heart rate of 72 bpm by 72 / 80=0.9 times in the time direction, as shown in FIG.

[0062] In this way, the pre-processing unit 51 normalizes the training time-varying image data with different sweep speeds and / or different heart rates to time-varying image data with a predetermined sweep speed and / or predetermined heart rate. The pre-processing unit 51 converts the time-varying image data normalized to the predetermined sweep speed and / or predetermined heart rate into time-varying image data with a predetermined width as shown in Fig. 13, and acquires training time-varying image data with a predetermined width in the time direction.

[0063] Furthermore, the preprocessing unit 51 may apply data extension to the time-variation image data normalized in the time direction, further transforming the data in the time direction. That is, in a case where the subject's heart rate cannot be obtained, the preprocessing unit 51 may handle differences in heart rate by applying data extension to the time-variation image data normalized according to the sweep speed, expanding or contracting the data in the time direction at a random ratio, as shown in FIG.

[0064] The training unit 52 uses the time-varying image data preprocessed in this manner to train the machine learning model 10 to be trained. Specifically, the training unit 52 adjusts parameters of the machine learning model 10, such as a convolutional neural network, using training data consisting of the time-varying image data for training normalized with respect to the sweep speed and / or heart rate and ground truth data including the detection target corresponding to the time-varying image data for training. Here, the ground truth data may be information about features in the time-varying image that are affected by time changes. Specifically, the ground truth data (i.e., label data) may be information about the position of one or more points in the time-varying image, information about the width, information about the waveform shape, information about time timing, information about the time interval, information about the trace, etc.

[0065] For example, if the machine learning model 10 to be trained is a convolutional neural network used for image processing, the training unit 52 may input normalized time-varying image data for training to the convolutional neural network to be trained, and adjust the parameters of the convolutional neural network to be trained according to the backpropagation algorithm based on the error between the output result from the convolutional neural network to be trained and the ground truth data. The training unit 52 continues the training process until a predetermined termination condition is satisfied, and terminates the training process when the predetermined termination condition is satisfied.

[0066] 15 is a block diagram showing the functional configuration of an ultrasound diagnostic apparatus 100 according to one embodiment of the present disclosure. As shown in FIG. 15, the ultrasound diagnostic apparatus 100 according to this embodiment includes an ultrasound control unit 110, a preprocessing unit 120, and an inference unit 130. For example, one or more functional units of the ultrasound control unit 110, the preprocessing unit 120, and the inference unit 130 may be realized by one or more CPUs 1171 executing one or more programs or instructions stored in a ROM 1172 or a RAM 1173.

[0067] The ultrasound control unit 110 controls the ultrasound signals transmitted and received by the ultrasound diagnostic apparatus 100. Specifically, the ultrasound control unit 110 controls the ultrasound signals transmitted from the ultrasound probe 1020 to the subject 30, and receives the ultrasound signals reflected from the subject 30. For example, the ultrasound control unit 110 may transmit ultrasound signals from the ultrasound probe 1020 and receive ultrasound signals reflected from an object. The ultrasound control unit 110 generates time-varying image data of the detection target region of the subject 30 based on the received ultrasound signals and at a sweep rate set by the user.

[0068] The preprocessing unit 120 preprocesses the time-varying image data. Specifically, if the trained machine learning model 10 has been trained using time-varying image data normalized by a predetermined sweep speed and / or a predetermined heart rate, the preprocessing unit 120 normalizes the time-varying image data acquired from the subject 30 by the ultrasound control unit 110 by the predetermined sweep speed and / or a predetermined heart rate. For example, as shown in FIG. 12 , if the time-varying image data acquired by the ultrasound control unit 110 has a sweep speed of 30 mm / sec and a heart rate of 72 bpm, the preprocessing unit 120 first enlarges the acquired time-varying image by 40 / 30=1.33·· times to normalize it to the predetermined sweep speed of 40 mm / sec, and then reduces it by 72 / 80=0.9 times to normalize it to the predetermined heart rate of 80 bpm. The preprocessing unit 120 passes the normalized time-varying image data to the inference unit 130.

[0069] The inference unit 130 uses the trained machine learning model 10 to obtain an inference result for the time-varying image data normalized by the preprocessing unit 120. Specifically, the inference unit 130 inputs the normalized time-varying image data to the trained machine learning model 10 and obtains an inference result for the detection target region from the trained machine learning model 10. The inference unit 130 performs processing on the inference result that is the reverse of the enlargement and / or reduction performed by the preprocessing unit 120, and obtains an inference result corresponding to the time-varying image data obtained by the ultrasound control unit 110.

[0070] [Example of VTI measurement] The machine learning model 10 according to this embodiment can be used for VTI (Velocity-Time Integral) measurement. Here, VTI stands for velocity time integral, and VTI in the left ventricular outflow tract (LVOT) of the heart is used as an evaluation index of left ventricular systolic function. The area of ​​a PW Doppler velocity waveform is equal to the time integral VTI of blood velocity. Here, the LVOT diameter can be measured, for example, in a left parasternal long-axis view, as shown in FIG. 16 . In FIG. 16 , the area of ​​the enclosed region on the time-varying image data corresponds to the VTI. For VTI measurement, the machine learning model 10 is trained to detect a VTI measurement target section and / or a VTI measurement target region, as shown in FIG. 17 , from input time-varying image data.

[0071] In the training process of the machine learning model 10, for example, time-varying image data as shown in FIG. 18 and label data of the measurement section corresponding to the time-varying image data are prepared as training data. Because the time-varying image data for training is acquired at different sweep speeds and / or different heart rates, the time-varying image data and label data are normalized to a predetermined sweep speed and / or a predetermined heart rate as described above, and are expanded and / or reduced in the time direction as shown. For example, the time-varying image data and label data shown in FIG. 18 are first reduced in the time direction to normalize to the predetermined sweep speed, and then expanded in the time direction to normalize to the predetermined heart rate. The time-varying image data and label data thus expanded and reduced are superimposed on a background image to have a predetermined width in the time direction. Then, the time-varying image data and label data having the predetermined width are input to the training process.

[0072] Next, in an inference process using the trained machine learning model 10, when time-varying image data of an inference target such as that shown in FIG. 19 is given, the time-varying image data of the inference target is normalized in the time direction and superimposed on a background image so as to have a predetermined width in the time direction. This normalized time-varying image data having a predetermined time width is input to the trained machine learning model 10, which predicts a VTI measurement section in the normalized time-varying image data as shown in FIG. 19. Then, a process opposite to the preprocessing is performed on the predicted VTI measurement section. That is, the portion of the background image superimposed in the preprocessing is removed from the VTI measurement section, and the VTI measurement section is then restored to its original width in the time direction. The VTI measurement section obtained in this manner may be displayed superimposed on the time-varying image data of the inference target.

[0073] 20 is a flowchart showing an ultrasound diagnostic process according to an embodiment of the present disclosure. As shown in Fig. 20, in step S101, the ultrasound diagnostic device 100 creates a left parasternal long axis view in B-mode and freezes the created image.

[0074] In step S102, the ultrasound diagnostic device 100 measures the LVOT diameter. For example, the ultrasound diagnostic device 100 may measure the length of a portion indicating the LVOT diameter specified by the user on the frozen left parasternal long-axis image.

[0075] In step S103, the ultrasound diagnostic apparatus 100 renders an apical long axis view (three-chamber view) in B mode and aligns the Doppler cursor with the LVOT. For example, the ultrasound diagnostic apparatus 100 receives an instruction for the LVOT from the user on the rendered apical long axis view.

[0076] In step S104, the ultrasound diagnostic apparatus 100 transitions to a PW (Pulsed Wave) Doppler mode.

[0077] In step S105, the ultrasound diagnostic device 100 acquires a PW Doppler velocity waveform and a Doppler trace of the LVOT, performs automatic VTI measurement on the PW Doppler velocity waveform as time-varying image data, and calculates and displays the VTI from the VTI measurement section and the Doppler trace. That is, the ultrasound diagnostic device 100 inputs the PW Doppler velocity waveform as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the VTI measurement section from the trained machine learning model 10. Then, the ultrasound diagnostic device 100 calculates the VTI from the acquired VTI measurement section and Doppler trace.

[0078] In step S106, the ultrasound diagnostic apparatus 100 determines whether or not the state has transitioned to the frozen state. If the state has not transitioned to the frozen state (S106: NO), the ultrasound diagnostic apparatus 100 proceeds to step S105 and repeats the above-mentioned processing. On the other hand, if the state has transitioned to the frozen state (S106: YES), the ultrasound diagnostic apparatus 100 proceeds to step S107.

[0079] In step S107, the ultrasound diagnostic device 100 performs automatic VTI measurement on the frozen PW Doppler velocity waveform, and calculates and displays the VTI from the VTI measurement section and the Doppler trace. That is, the ultrasound diagnostic device 100 inputs the PW Doppler velocity waveform as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the VTI measurement section from the trained machine learning model 10. The ultrasound diagnostic device 100 then calculates the VTI from the acquired VTI measurement section and Doppler trace.

[0080] In step S108, the ultrasound diagnostic apparatus 100 calculates the stroke volume (SV) from the LVOT diameter and VTI, and calculates the cardiac output (CO) from the SV and the heart rate (HR).

[0081] In this way, according to this embodiment, automatic VTI measurement is performed using the machine learning model 10 trained with training data consisting of normalized time-varying image data and a VTI measurement target section, and the VTI can be estimated from the time-varying image data acquired from the subject 30 based on the estimated VTI measurement target section and Doppler trace.

[0082] In the above-described embodiment, the VTI was estimated from the VTI measurement target section estimated by automatic VTI measurement and the Doppler trace, but the machine learning model 10 according to this embodiment may also be trained to detect the VTI measurement target area, as shown in FIG. 21.

[0083] In the training process of the machine learning model 10, for example, time-varying image data as shown in FIG. 22 and label data of the measurement region corresponding to the time-varying image data are prepared as training data. Because the time-varying image data for training is acquired at different sweep speeds and / or different heart rates, the time-varying image data and label data are normalized to a predetermined sweep speed and / or a predetermined heart rate as described above, and are expanded and / or reduced in the time direction as shown. For example, the time-varying image data and label data shown in FIG. 22 are first reduced in the time direction to normalize to the predetermined sweep speed, and then expanded in the time direction to normalize to the predetermined heart rate. The time-varying image data and label data thus expanded and reduced are superimposed on a background image to have a predetermined width in the time direction. Then, the time-varying image data and label data having the predetermined width are input to the training process.

[0084] Next, in an inference process using the trained machine learning model 10, when time-varying image data of an inference target such as that shown in FIG. 23 is given, the time-varying image data of the inference target is normalized in the time direction and superimposed on a background image so as to have a predetermined width in the time direction. This normalized time-varying image data having a predetermined time width is input to the trained machine learning model 10, which predicts a VTI measurement target region in the normalized time-varying image data as shown in FIG. 23. Then, a process opposite to the preprocessing is performed on the predicted VTI measurement target region. That is, the portion of the background image superimposed in the preprocessing is removed from the VTI measurement target region, and the region is then returned to its original width in the time direction. The VTI measurement region thus obtained may be superimposed on the time-varying image data of the inference target.

[0085] 24 is a flowchart showing ultrasound diagnostic processing according to an embodiment of the present disclosure. As shown in FIG. 24, in step S201, the ultrasound diagnostic device 100 creates a left parasternal long axis view in B-mode and freezes the created image.

[0086] In step S202, the ultrasound diagnostic device 100 measures the LVOT diameter. For example, the ultrasound diagnostic device 100 may measure the length of a portion indicating the LVOT diameter specified by the user on the frozen left parasternal long-axis image.

[0087] In step S203, the ultrasound diagnostic apparatus 100 renders an apical long axis view (three-chamber view) in B mode and aligns the Doppler cursor with the LVOT. For example, the ultrasound diagnostic apparatus 100 receives an instruction for the LVOT from the user on the rendered apical long axis view.

[0088] In step S204, the ultrasound diagnostic apparatus 100 transitions to the PW Doppler mode.

[0089] In step S205, the ultrasound diagnostic device 100 acquires a PW Doppler velocity waveform of the LVOT, performs automatic VTI measurement on the PW Doppler velocity waveform as time-varying image data, and calculates and displays the VTI from the estimated VTI measurement target region. That is, the ultrasound diagnostic device 100 inputs the PW Doppler velocity waveform as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the VTI measurement target region from the trained machine learning model 10. Then, the ultrasound diagnostic device 100 calculates the VTI from the acquired VTI measurement target region.

[0090] In step S206, the ultrasound diagnostic apparatus 100 determines whether the state has transitioned to the frozen state. If the state has not transitioned to the frozen state (S206: NO), the ultrasound diagnostic apparatus 100 proceeds to step S205 and repeats the above-mentioned processing. On the other hand, if the state has transitioned to the frozen state (S206: YES), the ultrasound diagnostic apparatus 100 proceeds to step S207.

[0091] In step S207, the ultrasound diagnostic device 100 performs automatic VTI measurement on the frozen PW Doppler velocity waveform, and calculates and displays the VTI from the estimated VTI measurement target region. That is, the ultrasound diagnostic device 100 inputs the PW Doppler velocity waveform as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the VTI measurement target region from the trained machine learning model 10. Then, the ultrasound diagnostic device 100 calculates the VTI from the acquired VTI measurement target region.

[0092] In step S208, the ultrasound diagnostic apparatus 100 calculates the stroke volume (SV) from the LVOT diameter and VTI, and calculates the cardiac output (CO) from the SV and the heart rate (HR).

[0093] In this way, according to this embodiment, automatic VTI measurement is performed using the machine learning model 10 trained with training data consisting of normalized time-varying image data and a VTI measurement target area, and the VTI can be estimated from the time-varying image data acquired from the subject 30 based on the estimated VTI measurement target area.

[0094] [Example of PSV / EDV] As shown in FIG. 25 , the machine learning model 10 according to this embodiment can be used to measure peak systolic velocity (PSV) and / or end diastolic velocity (EDV). Here, PSV is the maximum blood velocity during one heartbeat, and EDV is the blood velocity at end diastole (just before the heart contracts). PSV and EDV can be used to calculate the resistance index (RI) and pulsatility index (PI). That is, RI = (PSV - EDV) / PSV, and PI = (PSV - EDV) / TAMV. Here, time-averaged maximum velocity (TAMV) is the time average of the maximum velocity over one heartbeat. To measure PSV and EDV, the machine learning model 10 is trained to detect the position and / or timing of PSV and EDV from input time-varying image data.

[0095] In the training process of the machine learning model 10, for example, time-varying image data as shown in FIG. 26 and label data of PSV positions and EDV positions corresponding to the time-varying image data are prepared as training data. Because the time-varying image data for training is acquired at different sweep speeds and / or different heart rates, the time-varying image data and label data are normalized to a predetermined sweep speed and / or a predetermined heart rate as described above, and are expanded and / or reduced in the time direction as shown. For example, the time-varying image data and label data shown in FIG. 26 are first reduced in the time direction to normalize to a predetermined sweep speed, and then expanded in the time direction to normalize to a predetermined heart rate. The time-varying image data and label data thus expanded and reduced are superimposed on a background image to have a predetermined width in the time direction. Then, the time-varying image data and label data having the predetermined width are input to the training process.

[0096] Next, in an inference process using the trained machine learning model 10, when time-varying image data of an inference target such as that shown in FIG. 27 is given, the time-varying image data of the inference target is normalized in the time direction and superimposed on a background image so as to have a predetermined width in the time direction. This normalized time-varying image data having a predetermined time width is input to the trained machine learning model 10, which predicts the PSV position and EDV position in the normalized time-varying image data as shown in FIG. 27. Then, a process opposite to the preprocessing is performed on the predicted PSV position and EDV position. That is, the portion of the background image superimposed in the preprocessing is removed from the PSV position and EDV position, and then the original width is restored in the time direction. The PSV position and EDV position thus obtained may be superimposed on the time-varying image data of the inference target and displayed.

[0097] 28 is a flowchart showing ultrasound diagnostic processing according to an embodiment of the present disclosure. As shown in FIG. 28, in step S301, ultrasound diagnostic device 100 visualizes a blood vessel to be measured in B-mode and aligns a Doppler cursor with the blood vessel. For example, the user instructs ultrasound diagnostic device 100 on the visualized blood vessel to identify a blood vessel portion to be measured and the Doppler cursor.

[0098] In step S302, the ultrasound diagnostic apparatus 100 transitions to the PW Doppler mode.

[0099] In step S303, the ultrasound diagnostic apparatus 100 acquires a PW velocity waveform and a Doppler trace in a blood vessel, performs automatic PSV / EDV measurement on the PW Doppler velocity waveform as time-varying image data, and calculates and displays the flow velocities of PSV and EDV.

[0100] In step S304, the ultrasound diagnostic apparatus 100 calculates and displays a resistance coefficient RI from the PSV and EDV, calculates a time-averaged maximum blood velocity (TAMV) from the Doppler trace, and calculates and displays a pulsatility index (PI) from the PSV and TAMV.

[0101] In step S305, the ultrasound diagnostic apparatus 100 determines whether the state has transitioned to the frozen state. If the state has not transitioned to the frozen state (S305: NO), the ultrasound diagnostic apparatus 100 proceeds to step S303 and repeats the above-mentioned processing. On the other hand, if the state has transitioned to the frozen state (S305: YES), the ultrasound diagnostic apparatus 100 proceeds to step S306.

[0102] In step S306, the ultrasound diagnostic device 100 performs automatic PSV-EDV measurement on the frozen PW Doppler velocity waveform, and calculates and displays the flow velocities of the PSV and EDV from the PSV position and EDV position. That is, the ultrasound diagnostic device 100 inputs the PW Doppler velocity waveform as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the PSV position and EDV position from the trained machine learning model 10. The ultrasound diagnostic device 100 then calculates and displays the flow velocities of the PSV and EDV from the acquired PSV position and EDV position.

[0103] In step S307, the ultrasound diagnostic apparatus 100 calculates and displays a resistive index (RI) from the PSV and EDV, calculates a time-averaged maximum blood velocity (TAMV) from the Doppler trace, and calculates and displays a pulsatility index (PI) from the PSV and TAMV.

[0104] In this way, according to this embodiment, automatic PSV / EDV measurement is performed using the machine learning model 10 trained with training data consisting of normalized time-varying image data and PSV and EDV positions, and RI and PI can be estimated from the time-varying image data obtained from the subject 30 based on the estimated PSV and EDV positions, the Doppler trace, and the TAMV calculated from the Doppler trace.

[0105] In the above-described embodiment, RI and PI were estimated from the PSV and EDV positions estimated by automatic PSV / EDV measurement and the Doppler trace, but the machine learning model 10 according to this embodiment may also be trained to detect PSV timing and EDV timing, as shown in FIG. 29.

[0106] In the training process of the machine learning model 10, for example, time-varying image data as shown in FIG. 30 and label data of PSV timing and EDV timing corresponding to the time-varying image data are prepared as training data. Because the time-varying image data for training is acquired at different sweep speeds and / or different heart rates, the time-varying image data and label data are normalized to a predetermined sweep speed and / or a predetermined heart rate, as described above, and are expanded and / or reduced in the time direction as shown. For example, the time-varying image data and label data shown in FIG. 30 are first reduced in the time direction to normalize to a predetermined sweep speed, and then expanded in the time direction to normalize to a predetermined heart rate. The time-varying image data and label data thus expanded and reduced are superimposed on a background image to have a predetermined width in the time direction. Then, the time-varying image data and label data having a predetermined width are input to the training process.

[0107] Next, in an inference process using the trained machine learning model 10, when time-varying image data of an inference target such as that shown in FIG. 31 is given, the time-varying image data of the inference target is normalized in the time direction and superimposed on a background image so as to have a predetermined width in the time direction. This normalized time-varying image data having a predetermined time width is input to the trained machine learning model 10, which predicts the PSV timing and EDV timing in the normalized time-varying image data as shown in FIG. 31. Then, a process opposite to the preprocessing is performed on the predicted PSV timing and EDV timing. That is, the portion of the background image superimposed in the preprocessing is removed from the PSV timing and EDV timing, and then the original width is restored in the time direction. The PSV timing and EDV timing obtained in this manner may be superimposed on the time-varying image data of the inference target and displayed.

[0108] 32 is a flowchart showing ultrasound diagnostic processing according to an embodiment of the present disclosure. As shown in FIG. 32, in step S401, ultrasound diagnostic device 100 visualizes a blood vessel to be measured in B-mode and aligns a Doppler cursor with the blood vessel. For example, the ultrasound diagnostic device 100 receives instructions from the user regarding the blood vessel portion to be measured and the Doppler cursor on the visualized blood vessel.

[0109] In step S402, the ultrasound diagnostic apparatus 100 transitions to the PW Doppler mode.

[0110] In step S403, the ultrasound diagnostic device 100 acquires a PW velocity waveform and a Doppler trace in a blood vessel, performs automatic PSV / EDV measurement on the PW Doppler velocity waveform as time-varying image data, and calculates and displays the flow velocities of PSV and EDV from the estimated PSV timing and EDV timing and the Doppler trace. That is, the ultrasound diagnostic device 100 inputs the PW Doppler velocity waveform as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the PSV timing and EDV timing from the trained machine learning model 10. The ultrasound diagnostic device 100 then calculates and displays the flow velocities of PSV and EDV from the acquired PSV timing and EDV timing and the Doppler trace.

[0111] In step S404, the ultrasound diagnostic apparatus 100 calculates and displays a resistive index (RI) from the PSV and EDV, calculates a time-averaged maximum blood velocity (TAMV) from the Doppler trace, and calculates and displays a pulsatility index (PI) from the PSV and TAMV.

[0112] In step S405, the ultrasound diagnostic apparatus 100 determines whether or not the state has transitioned to the frozen state. If the state has not transitioned to the frozen state (S405: NO), the ultrasound diagnostic apparatus 100 proceeds to step S403 and repeats the above-mentioned processing. On the other hand, if the state has transitioned to the frozen state (S405: YES), the ultrasound diagnostic apparatus 100 proceeds to step S406.

[0113] In step S406, the ultrasound diagnostic device 100 performs automatic PSV / EDV measurement on the frozen PW Doppler velocity waveform, and calculates and displays the PSV and EDV flow velocities from the estimated PSV timing and EDV timing and the Doppler trace. That is, the ultrasound diagnostic device 100 inputs the PW Doppler velocity waveform as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the PSV timing and EDV timing from the trained machine learning model 10. The ultrasound diagnostic device 100 then calculates and displays the PSV and EDV flow velocities from the acquired PSV timing and EDV timing and the Doppler trace.

[0114] In step S407, the ultrasound diagnostic apparatus 100 calculates and displays a resistive index (RI) from the PSV and EDV, calculates a time-averaged maximum blood velocity (TAMV) from the Doppler trace, and calculates and displays a pulsatility index (PI) from the PSV and TAMV.

[0115] In this way, according to this embodiment, automatic PSV / EDV measurement is performed using the machine learning model 10 trained with training data consisting of normalized time-varying image data and PSV timing and EDV timing, and RI and PI can be estimated from the time-varying image data acquired from the subject 30 based on the estimated PSV timing and EDV timing, the Doppler trace, and the TAMV calculated from the Doppler trace.

[0116] [Example of blood flow volume] The machine learning model 10 according to this embodiment can be used to measure blood flow volume (FL), as shown in Fig. 33. Here, the blood flow volume per unit time FL [mL / min] is calculated from the time-averaged blood flow velocity TAV [cm / sec] per heartbeat measured by Doppler and the vascular cross-sectional area S [mm 2 ] can be calculated by multiplying S=π×(D / 2) 2 and FL=(S / 10 2) × (TAV × 60). For automatic blood flow measurement, the machine learning model 10 is trained to detect the boundary of the blood flow measurement target section (i.e., the section of one heartbeat) from the input time-varying image data. The boundary of the measurement target section shown in the figure is set based on the timing of EDV, but the present disclosure is not necessarily limited to this.

[0117] In the training process of the machine learning model 10, for example, time-varying image data as shown in FIG. 34 and label data of EDV timing corresponding to the time-varying image data are prepared as training data. Since the time-varying image data for training is acquired at different sweep speeds and / or different heart rates, the time-varying image data and label data are normalized to a predetermined sweep speed and / or a predetermined heart rate, as described above, and are expanded and / or reduced in the time direction. For example, the time-varying image data and label data are first reduced in the time direction to normalize to a predetermined sweep speed, and then expanded in the time direction to normalize to a predetermined heart rate. The time-varying image data and label data thus expanded and reduced are superimposed on a background image to have a predetermined width in the time direction. Then, the time-varying image data and label data having a predetermined width are input into the training process. Note that the normalization of the training data is similar to that in the embodiment related to the detection of EDV timing described above, and details will be omitted to avoid repetition.

[0118] Next, in the inference process using the trained machine learning model 10, when time-varying image data of the inference target is given, the time-varying image data of the inference target is normalized in the time direction and superimposed on the background image so as to have a predetermined width in the time direction. The normalized time-varying image data having a predetermined time width is input to the trained machine learning model 10, which predicts the EDV timing in the normalized time-varying image data. Then, the predicted EDV timing is subjected to a process reverse to the preprocessing. That is, the portion of the background image superimposed in the preprocessing is removed from the EDV timing, and then the original width is restored in the time direction. The EDV timing obtained in this manner may be superimposed on the time-varying image data of the inference target for display. Note that the normalization of the data to be inferred is similar to the embodiment relating to the detection of the EDV timing described above, and details thereof will be omitted to avoid repetition.

[0119] 35 is a flowchart showing ultrasound diagnostic processing according to an embodiment of the present disclosure. As shown in FIG. 35, in step S501, ultrasound diagnostic device 100 visualizes a blood vessel to be measured in B-mode and aligns a Doppler cursor with the blood vessel. For example, the ultrasound diagnostic device 100 receives instructions from the user on the visualized blood vessel regarding the blood vessel portion to be measured and the Doppler cursor.

[0120] In step S502, the ultrasound diagnostic apparatus 100 transitions to the PW Doppler mode.

[0121] In step S503, the ultrasound diagnostic apparatus 100 acquires a PW velocity waveform and a Doppler trace in the blood vessel and transitions to a freeze state.

[0122] In step S504, the ultrasound diagnostic device 100 performs automatic blood flow measurement on the PW Doppler velocity waveform as time-varying image data, and detects a measurement target section such as the estimated EDV timing. That is, the ultrasound diagnostic device 100 inputs the PW Doppler velocity waveform as time-varying image data to be measured to the trained machine learning model 10, and acquires the EDV timing from the trained machine learning model 10.

[0123] In step S505, the ultrasound diagnostic apparatus 100 time-averages the average flow velocity in the measurement section to obtain the time-averaged blood flow velocity TAV.

[0124] In step S506, the ultrasound diagnostic apparatus 100 measures the blood vessel diameter D on the B-mode image.

[0125] In step S507, the ultrasound diagnostic device 100 calculates the blood flow rate FL from the time average blood flow velocity TAV and the blood vessel diameter D. Specifically, the ultrasound diagnostic device 100 calculates the blood flow rate FL from S=π×(D / 2) 2 The cross-sectional area S of the blood vessel is calculated from the blood vessel diameter D according to the following formula: FL = (S / 10 2 The blood flow rate FL is calculated from the vascular cross-sectional area S and the time-averaged blood flow velocity TAV according to the formula: )×(TAV×60).

[0126] In this way, according to this embodiment, automatic blood flow measurement is performed using the machine learning model 10 trained with training data consisting of normalized time-varying image data and EDV timing, and blood flow can be estimated from the time-varying image data acquired from the subject 30 based on the estimated EDV timing and Doppler trace.

[0127] [Example of E / A ratio] The machine learning model 10 according to this embodiment can be used to measure the E / A ratio, as shown in FIG. 36 . Here, the E / A ratio is an evaluation index of left ventricular diastolic function and can be calculated as E / A ratio = maximum E wave velocity / maximum A wave velocity. For example, if E / A≦0.8, the left ventricle is determined to be of impaired relaxation type, and if E / A≧2.0, the left ventricle is determined to be of restrictive type. For automatic E / A ratio measurement, the machine learning model 10 is trained to detect the positions of maximum E wave velocity and maximum A wave velocity from input time-varying image data.

[0128] In the training process of the machine learning model 10, for example, time-varying image data such as that shown in FIG. 37 and label data of the positions of maximum E-wave velocity and maximum A-wave velocity corresponding to the time-varying image data are prepared as training data. Because the time-varying image data for training is acquired at different sweep speeds and / or different heart rates, the time-varying image data and label data are normalized to a predetermined sweep speed and / or a predetermined heart rate, as described above, and are expanded and / or reduced in the time direction. For example, the time-varying image data and label data are first reduced in the time direction to normalize to a predetermined sweep speed, and then expanded in the time direction to normalize to a predetermined heart rate. The time-varying image data and label data thus expanded and reduced are superimposed on a background image to have a predetermined width in the time direction. The time-varying image data and label data having a predetermined width are then input into the training process. Note that the normalization of the training data is similar to that in the embodiment relating to the detection of PSV and EDV positions described above, and details will be omitted to avoid repetition.

[0129] Next, in the inference process using the trained machine learning model 10, when time-varying image data of the inference target is given, the time-varying image data of the inference target is normalized in the time direction and superimposed on a background image so as to have a predetermined width in the time direction. The normalized time-varying image data having a predetermined time width is input to the trained machine learning model 10, which predicts the positions of the maximum E wave velocity and the maximum A wave velocity in the normalized time-varying image data. Then, the reverse process of the preprocessing is performed on the predicted positions of the maximum E wave velocity and the maximum A wave velocity. That is, the portions of the background image superimposed in the preprocessing are removed from the positions of the maximum E wave velocity and the maximum A wave velocity, and then the positions are returned to their original width in the time direction. The positions of the maximum E wave velocity and the maximum A wave velocity obtained in this manner may be superimposed and displayed on the time-varying image data of the inference target. Note that the normalization of the inference target data is similar to that in the embodiment relating to the detection of PSV and EDV positions described above, and details will be omitted to avoid repetition.

[0130] 38 is a flowchart showing ultrasound diagnostic processing according to an embodiment of the present disclosure. As shown in FIG. 38, in step S601, the ultrasound diagnostic device 100 renders an apical four-chamber view of the heart in B-mode and aligns the Doppler cursor between the tips of the mitral valve leaflets. For example, the ultrasound diagnostic device 100 receives instructions from the user regarding the position of the Doppler cursor on the rendered apical four-chamber view.

[0131] In step S602, the ultrasound diagnostic apparatus 100 transitions to the PW Doppler mode.

[0132] In step S603, the ultrasound diagnostic device 100 acquires a PW Doppler velocity waveform of the mitral valve inflow blood flow, performs automatic E / A measurement on the PW Doppler velocity waveform as time-varying image data, and acquires the maximum E-wave velocity and the maximum A-wave velocity. That is, the ultrasound diagnostic device 100 inputs the PW Doppler velocity waveform as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the positions of the maximum E-wave velocity and the maximum A-wave velocity from the trained machine learning model 10.

[0133] In step S604, the ultrasound diagnostic apparatus 100 calculates and displays the E / A ratio from the acquired maximum E wave velocity and maximum A wave velocity.

[0134] In step S605, the ultrasound diagnostic apparatus 100 determines whether the state has transitioned to the frozen state. If the state has not transitioned to the frozen state (S605: NO), the ultrasound diagnostic apparatus 100 proceeds to step S603 and repeats the above-mentioned processing. On the other hand, if the state has transitioned to the frozen state (S605: YES), the ultrasound diagnostic apparatus 100 proceeds to step S606.

[0135] In step S606, the ultrasound diagnostic device 100 performs automatic E / A measurement on the frozen PW Doppler velocity waveform to acquire the maximum E wave velocity and maximum A wave velocity. That is, the ultrasound diagnostic device 100 inputs the PW Doppler velocity waveform as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the positions of the maximum E wave velocity and the maximum A wave velocity from the trained machine learning model 10. The ultrasound diagnostic device 100 then acquires the maximum E wave velocity and the maximum A wave velocity from the acquired positions of the maximum E wave velocity and the maximum A wave velocity.

[0136] In step S607, the ultrasound diagnostic apparatus 100 calculates and displays the E / A ratio from the maximum E wave velocity and the maximum A wave velocity.

[0137] In this way, according to this embodiment, automatic E / A measurement is performed using the machine learning model 10 trained with training data consisting of normalized time-varying image data and the positions of maximum E wave velocity and maximum A wave velocity, and the E / A ratio can be estimated from the time-varying image data acquired from the subject 30 based on the estimated positions of maximum E wave velocity and maximum A wave velocity.

[0138] In the above-described embodiment, the E / A ratio was estimated from the position of the maximum E wave velocity and the position of the maximum A wave velocity estimated by automatic E / A measurement, but the machine learning model 10 according to this embodiment may also be trained to detect the timing of the maximum E wave velocity and the timing of the maximum A wave velocity, as shown in FIG. 39.

[0139] In the training process of the machine learning model 10, for example, time-varying image data as shown in FIG. 39 and label data of the maximum E-wave velocity timing and maximum A-wave velocity timing corresponding to the time-varying image data are prepared as training data. Because the time-varying image data for training is acquired at different sweep speeds and / or different heart rates, the time-varying image data and label data are normalized to a predetermined sweep speed and / or a predetermined heart rate, as described above, and are expanded and / or reduced in the time direction as shown. For example, the time-varying image data and label data are first reduced in the time direction to normalize to a predetermined sweep speed, and then expanded in the time direction to normalize to a predetermined heart rate. The time-varying image data and label data thus expanded and reduced are superimposed on a background image to have a predetermined width in the time direction. The time-varying image data and label data having a predetermined width are then input into the training process. Note that the normalization of the training data is similar to that in the embodiment relating to the detection of PSV and EDV timing described above, and details will be omitted to avoid repetition.

[0140] Next, in the inference process using the trained machine learning model 10, when time-varying image data of an inference target is given, the time-varying image data of the inference target is normalized in the time direction and superimposed on a background image so as to have a predetermined width in the time direction. The normalized time-varying image data having a predetermined time width is input to the trained machine learning model 10, which predicts the maximum E wave velocity timing and maximum A wave velocity timing in the normalized time-varying image data. Then, the reverse process of the preprocessing is performed on the predicted maximum E wave velocity timing and maximum A wave velocity timing. That is, the background image portion superimposed in the preprocessing is removed from the maximum E wave velocity timing and maximum A wave velocity timing, and then the original width is restored in the time direction. The maximum E wave velocity timing and maximum A wave velocity timing obtained in this manner may be superimposed and displayed on the time-varying image data of the inference target. Note that the normalization of the inference target data is similar to that in the embodiment relating to the detection of PSV and EDV timing described above, and details thereof will be omitted to avoid repetition.

[0141] 40 is a flowchart showing ultrasound diagnostic processing according to one embodiment of the present disclosure. As shown in FIG. 40, in step S701, the ultrasound diagnostic device 100 renders an apical four-chamber view of the heart in B-mode and aligns the Doppler cursor between the tips of the mitral valve leaflets. For example, the ultrasound diagnostic device 100 receives instructions from the user regarding the position of the Doppler cursor on the rendered apical four-chamber view.

[0142] In step S702, the ultrasound diagnostic apparatus 100 transitions to the PW Doppler mode.

[0143] In step S703, the ultrasound diagnostic device 100 acquires a PW Doppler velocity waveform of the mitral valve inflow blood flow, performs automatic E / A measurement on the PW Doppler velocity waveform as time-varying image data, and acquires the maximum E-wave velocity timing and the maximum A-wave velocity timing. That is, the ultrasound diagnostic device 100 inputs the PW Doppler velocity waveform as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the maximum E-wave velocity timing and the maximum A-wave velocity timing from the trained machine learning model 10. Then, the maximum E-wave velocity and the maximum A-wave velocity are acquired from the maximum E-wave velocity timing, the maximum A-wave velocity timing, and the Doppler trace.

[0144] In step S704, the ultrasound diagnostic apparatus 100 calculates and displays the E / A ratio from the acquired maximum E wave velocity and maximum A wave velocity.

[0145] In step S705, the ultrasound diagnostic apparatus 100 determines whether the state has transitioned to the frozen state. If the state has not transitioned to the frozen state (S705: NO), the ultrasound diagnostic apparatus 100 proceeds to step S703 and repeats the above-mentioned processing. On the other hand, if the state has transitioned to the frozen state (S705: YES), the ultrasound diagnostic apparatus 100 proceeds to step S706.

[0146] In step S706, the ultrasound diagnostic device 100 performs automatic E / A measurement on the frozen PW Doppler velocity waveform and acquires the maximum E wave velocity timing and maximum A wave velocity timing. That is, the ultrasound diagnostic device 100 inputs the PW Doppler velocity waveform as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the maximum E wave velocity timing and maximum A wave velocity timing from the trained machine learning model 10. Then, the ultrasound diagnostic device 100 acquires the maximum E wave velocity and maximum A wave velocity from the acquired maximum E wave velocity timing and maximum A wave velocity timing and the Doppler trace.

[0147] In step S707, the ultrasound diagnostic apparatus 100 calculates and displays the E / A ratio from the maximum E wave velocity and the maximum A wave velocity.

[0148] In this way, according to this embodiment, automatic E / A measurement is performed using the machine learning model 10 trained with training data consisting of normalized time-varying image data and maximum E wave velocity timing and maximum A wave velocity timing, and the E / A ratio can be estimated from the time-varying image data acquired from the subject 30 based on the estimated maximum E wave velocity timing and maximum A wave velocity timing.

[0149] [Example of TAPSE / MAPSE] The machine learning model 10 according to this embodiment can be used to measure tricuspid annular plane systolic excursion (TAPSE) and / or mitral annular plane systolic excursion (MAPSE). TAPSE is an evaluation index of right ventricular function and is used to evaluate the amount of tricuspid annular movement in M-mode, as shown in FIG. 41 . MAPSE is an evaluation function of left ventricular function and can be used in cases where visualization of the left ventricle is difficult and is used to evaluate the amount of mitral annular movement in M-mode. As shown in FIG. 42 , TAPSE can be determined based on the diastolic and systolic positions in an M-mode image. Although not shown, MAPSE can also be determined similarly. For automatic TAPSE and MAPSE measurement, the machine learning model 10 is trained to detect the diastolic and systolic positions of TAPSE and / or MAPSE from input time-varying image data.

[0150] In the training process of the machine learning model 10, for example, time-varying image data as shown in FIG. 43 and label data of the position of the annulus during diastole and the position of the annulus during systole corresponding to the time-varying image data are prepared as training data. Because the time-varying image data for training is acquired at different sweep speeds and / or different heart rates, the time-varying image data and label data are normalized to a predetermined sweep speed and / or a predetermined heart rate, as described above, and are expanded and / or reduced in the time direction. For example, the time-varying image data and label data shown in FIG. 43 are first reduced in the time direction to normalize to a predetermined sweep speed, and then expanded in the time direction to normalize to a predetermined heart rate. The time-varying image data and label data thus expanded and reduced are superimposed on a background image to have a predetermined width in the time direction. The time-varying image data and label data having the predetermined width are then input into the training process.

[0151] Next, in the inference process using the trained machine learning model 10, for example, as shown in FIG. 44, when time-varying image data of an inference target is provided, the time-varying image data of the inference target is normalized in the time direction and superimposed on a background image so as to have a predetermined width in the time direction. The normalized time-varying image data having a predetermined time width is input to the trained machine learning model 10, which predicts the positions of the annulus during diastole and the annulus during systole in the normalized time-varying image data. Then, the reverse process of the preprocessing is performed on the predicted positions of the annulus during diastole and the annulus during systole. That is, the portions of the background image superimposed in the preprocessing are removed from the positions of the annulus during diastole and the annulus during systole, and then the original width is restored in the time direction. The positions of the annulus during diastole and the annulus during systole obtained in this manner may be superimposed on the time-varying image data of the inference target.

[0152] 45 is a flowchart showing ultrasound diagnostic processing according to an embodiment of the present disclosure. As shown in FIG. 45, in step S801, ultrasound diagnostic device 100 renders an apical four-chamber view of the heart in B-mode and aligns an M-mode cursor with the tricuspid valve annulus. For example, the ultrasound diagnostic device 100 receives an instruction from the user to position the M-mode cursor on the rendered apical four-chamber view.

[0153] In step S802, the ultrasound diagnostic apparatus 100 transitions to the M mode.

[0154] In step S803, the ultrasound diagnostic device 100 acquires an M-mode image of the tricuspid annulus and performs TAPSE automatic measurement on the M-mode image as time-varying image data to acquire the diastolic and systolic positions of the tricuspid annulus. That is, the ultrasound diagnostic device 100 inputs the M-mode image as time-varying image data to be measured to the trained machine learning model 10, and acquires the diastolic and systolic positions of the tricuspid annulus from the trained machine learning model 10.

[0155] In step S804, the ultrasound diagnostic apparatus 100 calculates and displays the TAPSE value from the acquired diastolic and systolic positions of the tricuspid annulus.

[0156] In step S805, the ultrasound diagnostic apparatus 100 determines whether the state has transitioned to the frozen state. If the state has not transitioned to the frozen state (S805: NO), the ultrasound diagnostic apparatus 100 proceeds to step S803 and repeats the above-mentioned processing. On the other hand, if the state has transitioned to the frozen state (S805: YES), the ultrasound diagnostic apparatus 100 proceeds to step S806.

[0157] In step S806, the ultrasound diagnostic device 100 performs automatic TAPSE measurement on the frozen M-mode image to acquire the diastolic and systolic positions of the tricuspid annulus. That is, the ultrasound diagnostic device 100 inputs the M-mode image as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the diastolic and systolic positions of the tricuspid annulus from the trained machine learning model 10.

[0158] In step S807, the ultrasound diagnostic apparatus 100 calculates and displays the TAPSE value from the acquired diastolic and systolic positions of the tricuspid annulus.

[0159] In this way, according to this embodiment, automatic TAPSE measurement is performed using the machine learning model 10 trained with training data consisting of normalized time-varying image data and the diastolic and systolic positions of the tricuspid annulus, and the TAPSE value can be estimated from the time-varying image data acquired from the subject 30 based on the estimated diastolic and systolic positions of the tricuspid annulus.

[0160] Regarding MAPSE, the MAPSE value can be similarly obtained using a machine learning model 10 that has been trained to estimate the diastolic and systolic positions of the mitral valve annulus in an M-mode image.

[0161] In the above-described embodiment, the TAPSE value was estimated from the diastolic and systolic positions of the tricuspid annulus estimated by automatic TAPSE measurement, but the machine learning model 10 according to this embodiment may also be trained to detect the diastolic and systolic timings of the tricuspid annulus, as shown in FIG. 46.

[0162] In the training process of the machine learning model 10, for example, time-varying image data as shown in FIG. 47 and label data of diastolic timing and systolic timing corresponding to the time-varying image data are prepared as training data. Because the time-varying image data for training is acquired at different sweep speeds and / or different heart rates, the time-varying image data and label data are normalized to a predetermined sweep speed and / or a predetermined heart rate, as described above, and are expanded and / or reduced in the time direction. For example, the time-varying image data and label data shown in FIG. 47 are first reduced in the time direction to normalize to a predetermined sweep speed, and then expanded in the time direction to normalize to a predetermined heart rate. The time-varying image data and label data thus expanded and reduced are superimposed on a background image to have a predetermined width in the time direction. Then, the time-varying image data and label data having the predetermined width are input to the training process.

[0163] Next, in an inference process using the trained machine learning model 10, for example, as shown in FIG. 48, when time-varying image data of an inference target is provided, the time-varying image data of the inference target is normalized in the time direction and superimposed on a background image so as to have a predetermined width in the time direction. The normalized time-varying image data having a predetermined time width is input to the trained machine learning model 10, which predicts the diastolic timing and the systolic timing in the normalized time-varying image data. Then, a process opposite to the preprocessing is performed on the predicted diastolic timing and the systolic timing. That is, the portion of the background image superimposed in the preprocessing is removed from the diastolic timing and the systolic timing, and then the original width is restored in the time direction. The diastolic timing and the systolic timing obtained in this manner may be superimposed on the time-varying image data of the inference target and displayed.

[0164] Furthermore, the machine learning model 10 according to this embodiment may be trained to detect a trace line of the tricuspid annulus in an M-mode image, as shown in FIG. 49. In the training process of the machine learning model 10, for example, time-varying image data as shown in FIG. 49 and label data of the trace line of the tricuspid annulus corresponding to the time-varying image data are prepared as training data. Since the time-varying image data for training is acquired at different sweep speeds and / or different heart rates, the time-varying image data and label data are normalized to a predetermined sweep speed and / or a predetermined heart rate, as described above, and expanded and / or contracted in the time direction. For example, the time-varying image data and label data shown in FIG. 48 are first contracted in the time direction to normalize to a predetermined sweep speed, and then expanded in the time direction to normalize to a predetermined heart rate. The time-varying image data and label data thus expanded and contracted are superimposed on a background image to have a predetermined width in the time direction. Then, the time-varying image data and label data having a predetermined width are input to the training process.

[0165] Next, in the inference process using the trained machine learning model 10, for example, as shown in FIG. 50, when time-varying image data of an inference target is provided, the time-varying image data of the inference target is normalized in the time direction and superimposed on a background image so as to have a predetermined width in the time direction. The normalized time-varying image data having a predetermined time width is input to the trained machine learning model 10, which predicts the position of a trace line of the tricuspid valve annulus in the normalized time-varying image data. Then, the reverse process of the preprocessing is performed on the predicted trace line of the tricuspid valve annulus. That is, the portion of the background image superimposed in the preprocessing is removed from the trace line of the tricuspid valve annulus, and the trace line is then restored to its original width in the time direction. The trace line of the tricuspid valve annulus obtained in this manner may be displayed superimposed on the time-varying image data of the inference target.

[0166] 51 is a flowchart showing ultrasound diagnostic processing according to one embodiment of the present disclosure. As shown in FIG. 51, in step S901, ultrasound diagnostic device 100 renders an apical four-chamber view of the heart in B-mode and aligns an M-mode cursor with the tricuspid valve annulus. For example, the ultrasound diagnostic device 100 receives instructions from the user regarding the position of the M-mode cursor on the rendered apical four-chamber view.

[0167] In step S902, the ultrasound diagnostic apparatus 100 transitions to the M mode.

[0168] In step S903, the ultrasound diagnostic device 100 acquires an M-mode image of the tricuspid annulus and performs TAPSE automatic measurement on the M-mode image as time-varying image data to acquire the diastolic timing and systolic timing of the tricuspid annulus and a trace line. That is, the ultrasound diagnostic device 100 inputs the M-mode image as time-varying image data to be measured to the trained machine learning model 10, and acquires the diastolic timing and systolic timing of the tricuspid annulus and a trace line from the trained machine learning model 10.

[0169] In step S904, the ultrasound diagnostic apparatus 100 acquires the diastolic and systolic positions from the trace lines at the acquired diastolic and systolic timings of the tricuspid annulus, and calculates and displays the TAPSE value from the diastolic and systolic positions.

[0170] In step S905, the ultrasound diagnostic apparatus 100 determines whether the state has transitioned to the frozen state. If the state has not transitioned to the frozen state (S905: NO), the ultrasound diagnostic apparatus 100 proceeds to step S903 and repeats the above-mentioned processing. On the other hand, if the state has transitioned to the frozen state (S905: YES), the ultrasound diagnostic apparatus 100 proceeds to step S906.

[0171] In step S906, the ultrasound diagnostic device 100 performs automatic TAPSE measurement on the frozen M-mode image to acquire the diastolic and systolic timings of the tricuspid annulus and a trace line. That is, the ultrasound diagnostic device 100 inputs the M-mode image as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the diastolic and systolic timings of the tricuspid annulus and a trace line from the trained machine learning model 10.

[0172] In step S907, the ultrasound diagnostic apparatus 100 acquires the diastolic and systolic positions from the trace lines at the acquired diastolic and systolic timings of the tricuspid annulus, and calculates and displays the TAPSE value from the diastolic and systolic positions.

[0173] In this way, according to this embodiment, automatic TAPSE measurement is performed using the machine learning model 10 trained with training data consisting of normalized time-varying image data and the diastolic and systolic timings of the tricuspid annulus and the trace line, and the TAPSE value can be estimated from the time-varying image data acquired from the subject 30 based on the estimated diastolic and systolic timings of the tricuspid annulus and the trace line.

[0174] Regarding MAPSE, the MAPSE value can be similarly obtained using a machine learning model 10 trained to estimate the diastolic and systolic timing and trace line of the mitral annulus in an M-mode image.

[0175] [Example of IVC diameter] The machine learning model 10 according to this embodiment can be used to measure the inferior vena cava (IVC). Here, the IVC diameter can be used to measure the maximum vascular diameter of the inferior vena cava and respiratory variation associated with breathing on a B-mode image or an M-mode image. The respiratory variation in vascular diameter can be expressed, for example, as (maximum diameter - minimum diameter) / maximum diameter x 100 [%]. As shown in FIG. 52 , the maximum diameter is the diameter of the inferior vena cava during expiration, and the minimum diameter is the diameter of the inferior vena cava during inspiration. The IVC diameter can be used, for example, as an index for infusion management. For automatic IVC diameter measurement, the machine learning model 10 is trained to detect the position and width of the blood vessel during expiration and the position and width of the blood vessel during inspiration from input time-varying image data.

[0176] In the training process of the machine learning model 10, for example, time-varying image data as shown in FIG. 53 and label data of the position and width of blood vessels during exhalation and the position and width of blood vessels during inhalation corresponding to the time-varying image data are prepared as training data. Because the time-varying image data for training is acquired at different sweep speeds, the time-varying image data and label data are normalized to a predetermined sweep speed and expanded and / or contracted in the time direction, as described above. Note that, because the IVC diameter is a measurement of respiratory variation rather than heartbeat, normalization according to heart rate is not performed. For example, the time-varying image data and label data shown in FIG. 53 are first contracted in the time direction to normalize to a predetermined sweep speed. Next, data expansion may be performed, expanding or contracting the data at a random rate in the time direction to accommodate different respiratory rates. The expanded and contracted time-varying image data and label data are superimposed on a background image to have a predetermined width in the time direction. The time-varying image data and label data having the predetermined width are then input to the training process.

[0177] Next, in the inference process using the trained machine learning model 10, for example, as shown in FIG. 54, when time-varying image data of an inference target is provided, the time-varying image data of the inference target is normalized in the time direction and superimposed on a background image so as to have a predetermined width in the time direction. The normalized time-varying image data having a predetermined time width is input to the trained machine learning model 10, which predicts the position and width of the blood vessels at exhalation and the position and width of the blood vessels at inhalation in the normalized time-varying image data. Then, a process opposite to the preprocessing is performed on the predicted position and width of the blood vessels at exhalation and the position and width of the blood vessels at inhalation. That is, the portion of the background image superimposed in the preprocessing is removed from the position and width of the blood vessels at exhalation and the position and width of the blood vessels at inhalation, and then the original width is restored in the time direction. The position and width of the blood vessels at exhalation and the position and width of the blood vessels at inhalation thus obtained may be superimposed and displayed on the time-varying image data of the inference target.

[0178] 55 is a flowchart showing ultrasound diagnostic processing according to one embodiment of the present disclosure. As shown in FIG. 55, in step S1001, the ultrasound diagnostic device 100 renders a long-axis image of the IVC in B-mode and aligns the M-mode cursor with the position of the IVC to be measured. For example, the ultrasound diagnostic device 100 receives an instruction from the user regarding the position of the M-mode cursor on the rendered long-axis image of the IVC.

[0179] In step S1002, the ultrasound diagnostic apparatus 100 transitions to the M mode.

[0180] In step S1003, the ultrasound diagnostic apparatus 100 acquires an M-mode image of the IVC, performs automatic IVC diameter measurement on the M-mode image as time-varying image data, and acquires the vascular diameter of the IVC during expiration and inspiration.

[0181] In step S1004, the ultrasound diagnostic apparatus 100 calculates and displays the respiratory variation of the IVC diameter from the acquired IVC diameters during expiration and inspiration.

[0182] In step S1005, the ultrasound diagnostic apparatus 100 determines whether the state has transitioned to the frozen state. If the state has not transitioned to the frozen state (S1005: NO), the ultrasound diagnostic apparatus 100 proceeds to step S1003 and repeats the above-mentioned processing. On the other hand, if the state has transitioned to the frozen state (S1005: YES), the ultrasound diagnostic apparatus 100 proceeds to step S1006.

[0183] In step S1006, the ultrasound diagnostic device 100 performs automatic IVC diameter measurement on the frozen M-mode image and acquires the IVC vascular diameters during expiration and inspiration. That is, the ultrasound diagnostic device 100 inputs the M-mode image as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the position and width of the blood vessel during expiration and the position and width of the blood vessel during inspiration from the trained machine learning model 10.

[0184] In step S1007, the ultrasound diagnostic apparatus 100 calculates and displays respiratory variations in the IVC diameter from the acquired IVC vascular diameters during expiration and inspiration.

[0185] In this way, according to this embodiment, automatic IVC diameter measurement is performed using the machine learning model 10 trained with normalized time-varying image data and training data consisting of the position and width of the blood vessel during expiration and the position and width of the blood vessel during inspiration, and respiratory fluctuations in the IVC diameter can be estimated from the time-varying image data acquired from the subject 30 based on the estimated position and width of the blood vessel during expiration and the position and width of the blood vessel during inspiration.

[0186] In the above-described embodiment, the respiratory variation of the IVC diameter was estimated from the position and width of the blood vessel during expiration and the position and width of the blood vessel during inspiration, which were estimated by automatic IVC diameter measurement. However, the machine learning model 10 according to this embodiment may also be trained to detect the timing of expiration and inspiration of the IVC and the anterior and posterior wall tracings, as shown in FIG. 56.

[0187] In the training process of the machine learning model 10, for example, time-varying image data and label data of the IVC at the time of expiration and inspiration corresponding to the time-varying image data are prepared as training data. Because the time-varying image data for training is acquired at different sweep rates, the time-varying image data and label data are normalized to a predetermined sweep rate and expanded and / or reduced in the time direction, as described above. For example, the time-varying image data and label data are first reduced in the time direction to normalize them to a predetermined sweep rate. Next, to accommodate different breathing rates, data expansion may be performed, in which the data is expanded or reduced at a random ratio in the time direction. The time-varying image data and label data expanded and reduced in this manner are superimposed on a background image to have a predetermined width in the time direction. Then, the time-varying image data and label data having the predetermined width are input to the training process. Note that the normalization of the training data is similar to that in the embodiment relating to the detection of diastolic and systolic timing in the TAPSE / MAPSE measurement described above, and details thereof will be omitted to avoid repetitive explanation. However, since the IVC diameter is a measurement of respiratory variation rather than heartbeat, it is not normalized according to heart rate.

[0188] Next, in the inference process using the trained machine learning model 10, for example, when time-varying image data of an inference target is provided, the time-varying image data of the inference target is normalized in the time direction and superimposed on a background image so as to have a predetermined width in the time direction. The normalized time-varying image data having a predetermined time width is input to the trained machine learning model 10, which predicts the IVC expiration and inspiration timings in the normalized time-varying image data. Then, the reverse process of the preprocessing is performed on the predicted IVC expiration and inspiration timings. That is, the background image portion superimposed in the preprocessing is removed from the IVC expiration and inspiration timings, and then the original width is restored in the time direction. The IVC expiration and inspiration timings acquired in this manner may be superimposed and displayed on the time-varying image data of the inference target. Note that the normalization of the inference target data is similar to the embodiment relating to the detection of diastolic and systolic timings in the TAPSE / MAPSE measurement described above, and details thereof will be omitted to avoid repetition.

[0189] Furthermore, the machine learning model 10 according to this embodiment may be trained to detect, for example, trace lines of the anterior and posterior walls in an M-mode image. In the training process of the machine learning model 10, for example, time-varying image data and label data of the trace lines of the anterior and posterior walls corresponding to the time-varying image data are prepared as training data. Because the time-varying image data for training is acquired at different sweep speeds, the time-varying image data and label data are normalized to a predetermined sweep speed and enlarged and / or reduced in the time direction, as described above. For example, the time-varying image data and label data are first reduced in the time direction to normalize to the predetermined sweep speed. The time-varying image data and label data thus enlarged and reduced are superimposed on a background image to have a predetermined width in the time direction. Then, the time-varying image data and label data having the predetermined width are input to the training process. Note that the normalization of the training data is similar to that in the embodiment relating to the detection of trace lines of the tricuspid annulus / mitral annulus in the TAPSE / MAPSE measurement described above, and details thereof will be omitted to avoid redundant description. However, since the IVC diameter is a measurement of respiratory variation rather than heartbeat, it is not normalized according to heart rate.

[0190] Next, in the inference process using the trained machine learning model 10, for example, when time-varying image data of an inference target is provided, the time-varying image data of the inference target is normalized in the time direction and superimposed on a background image so as to have a predetermined width in the time direction. The normalized time-varying image data having a predetermined time width is input to the trained machine learning model 10, which predicts the anterior and posterior wall trace lines in the normalized time-varying image data. Then, the reverse process of the preprocessing is performed on the predicted anterior and posterior wall trace lines. That is, the portions of the background image superimposed in the preprocessing are removed from the anterior and posterior wall trace lines, and then the trace lines are returned to their original width in the time direction. The anterior and posterior wall trace lines acquired in this manner may be superimposed and displayed on the time-varying image data of the inference target. Note that the normalization of the inference data is similar to the embodiment relating to the detection of the trace lines of the tricuspid valve annulus / mitral valve annulus in the TAPSE / MAPSE measurement described above, and details thereof will be omitted to avoid repetition.

[0191] 57 is a flowchart showing ultrasound diagnostic processing according to one embodiment of the present disclosure. As shown in FIG. 57, in step S1101, the ultrasound diagnostic device 100 renders a long-axis image of the IVC in B-mode and aligns the M-mode cursor with the position of the IVC to be measured. For example, the ultrasound diagnostic device 100 receives an instruction from the user regarding the position of the M-mode cursor on the rendered long-axis image of the IVC.

[0192] In step S1102, the ultrasound diagnostic apparatus 100 transitions to the M mode.

[0193] In step S1103, the ultrasound diagnostic apparatus 100 acquires an M-mode image of the IVC and performs automatic IVC diameter measurement on the M-mode image as time-varying image data, thereby acquiring the timing of expiration and inspiration and trace lines of the anterior and posterior walls of the IVC.

[0194] In step S1104, the ultrasound diagnostic apparatus 100 acquires the IVC diameter during expiration and inspiration from the anterior and posterior wall trace lines at the acquired expiration and inspiration timings, and calculates and displays the respiratory variation from the IVC diameter. That is, the distance between the anterior and posterior wall trace lines at the acquired expiration and inspiration timings is the IVC diameter.

[0195] In step S1105, the ultrasound diagnostic apparatus 100 determines whether the state has transitioned to the frozen state. If the state has not transitioned to the frozen state (S1105: NO), the ultrasound diagnostic apparatus 100 proceeds to step S1103 and repeats the above-mentioned processing. On the other hand, if the state has transitioned to the frozen state (S1105: YES), the ultrasound diagnostic apparatus 100 proceeds to step S1106.

[0196] In step S1106, the ultrasound diagnostic device 100 performs automatic IVC diameter measurement on the frozen M-mode image and acquires the timing of expiration and inspiration and the trace lines of the anterior and posterior walls of the IVC. That is, the ultrasound diagnostic device 100 inputs the M-mode image as time-varying image data of the measurement target to the trained machine learning model 10, and acquires the timing of expiration and inspiration and the trace lines of the anterior and posterior walls of the IVC from the trained machine learning model 10.

[0197] In step S1107, the ultrasound diagnostic apparatus 100 acquires the IVC diameter during expiration and inspiration from the trace lines of the anterior and posterior walls at the acquired timings of expiration and inspiration, and calculates and displays the respiratory variation from the IVC diameter. That is, the distance between the trace lines of the anterior and posterior walls at the acquired timings of expiration and inspiration is the IVC diameter.

[0198] In this way, according to this embodiment, automatic IVC diameter measurement is performed using the machine learning model 10 trained with training data consisting of normalized time-varying image data and timing of exhalation and inhalation and trace lines of the anterior and posterior walls, and respiratory fluctuations in IVC diameter can be estimated from the time-varying image data acquired from the subject 30 based on the estimated timing of exhalation and inhalation and trace lines of the anterior and posterior walls.

[0199] According to the above-described embodiment, a machine learning model is trained using training data including pairs of training time-varying image data, which is composed of time-varying image data normalized by expanding or contracting in the time direction time-varying image data based on received signals for image generation received by an ultrasound probe and / or time-varying image data generated based on the normalized time-varying image data, and ground truth data including a detection target corresponding to the training time-varying image data. For example, the machine learning model may be realized by any type of neural network, such as a convolutional neural network. Furthermore, the time-varying image data based on signals received by the ultrasound probe may be a Doppler image, an M-mode image, or the like, and the time-varying image data normalized from the time-varying image data may be time-varying image data normalized to a predetermined sweep speed. Furthermore, the time-varying image data generated based on the normalized time-varying image data may be time-varying image data normalized to a predetermined heart rate.

[0200] Furthermore, the detection target by the machine learning model may be information about features that are affected by time changes in the time-varying image. Specifically, the detection target may be information about the position of one or more points in the time-varying image, information about the width, information about the waveform shape, information about time timing, information about the time interval, information about the trace, etc. For example, when a Doppler image is acquired by an ultrasound diagnostic device, the machine learning model may detect the following from the acquired Doppler image: i) The time domain of the spectral waveform or the region of the spectral waveform that is the target of VTI (Velocity-Time Integral) measurement, ii) Position or timing of PSV (Peak Systolic Velocity) and EDV (End Diastolic Velocity) on the time axis, iii) The boundary of the time section of the spectral waveform that is the subject of blood flow measurement, or iv) The position or timing of the blood flow velocity of the E wave and the A wave on the time axis in E / A measurement, can be trained to detect as a detection target.

[0201] In addition, when an M-mode image is acquired by an ultrasound diagnostic device, the machine learning model can derive the following from the acquired M-mode image: i) The diastolic and systolic positions of the annulus in a TAPSE (Tricuspid Annular Plane Systolic Excursion) measurement or a MAPSE (Mitral Annular Plane Systolic Excursion) measurement, the timing of diastole and systole on the time axis, or a tracing of the annulus, or ii) Inferior vena cava (IVC) diameter measurement in M-mode, timing of expiration and inspiration, or tracing of the anterior and posterior walls of the inferior vena cava can be trained to detect as a detection target.

[0202] Although the examples of the present disclosure have been described in detail above, the present disclosure is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present disclosure as set forth in the claims. [Explanation of symbols]

[0203] 10 Machine Learning Models 20 Training Database (DB) 30 Subjects 40 Model Database (DB) 50 training equipment 51 Pre-processing section 52 Training Department 100 Ultrasound diagnostic equipment 110 Ultrasonic control unit 120 Pre-processing section 130 Reasoning part 200 Image processing device

Claims

1. At least one of training time-varying image data, which is a second time-varying image data obtained by normalizing the first time-varying image data based on the received signal for image generation received by the ultrasound probe in the time direction, and a third time-varying image data obtained by performing data extension to further transform the second time-varying image data in the time direction; Correct answer data including a detection target corresponding to the at least one training time-varying image data; a machine learning model trained using training data including pairs of normalizing the first time-varying image data in the time direction by processing the first time-varying image data based on a sweep speed when generating the first time-varying image data so that an image of the first time-varying image data, the image having a width direction in the time axis direction, is enlarged and / or reduced; Computer, A trained machine learning model characterized by being operated to output a detection target as an inference result from first predicted target time-varying image data, which is obtained by normalizing second predicted target time-varying image data based on a received signal for image generation received by an ultrasound probe in the time direction.

2. The machine learning model according to claim 1 , wherein the detection target is information about features affected by time changes in a time-varying image.

3. The machine learning model of claim 1 , wherein the detection target is at least one of information regarding the position of one or more points in a time-varying image, information regarding a width, information regarding a waveform shape, information regarding time timing, information regarding a time interval, and information regarding a trace.

4. The machine learning model according to claim 1 , wherein the training time-varying image data is time-varying image data cut out to a predetermined image width, or time-varying image data pasted onto background image data of a predetermined image width.

5. the first time-varying image data is a Doppler image; The detection target is i) A time section of a spectral waveform or a region of the spectral waveform that is the subject of VTI (Velocity-Time Integral) measurement; ii) Position or timing of PSV (Peak Systolic Velocity) and EDV (End Diastolic Velocity) on the time axis; iii) The boundary of the time section of the spectral waveform that is the target of blood flow measurement, or iv) The position or timing of the blood flow velocity of the E wave and the A wave on the time axis in the E / A measurement; The machine learning model according to claim 1, wherein the machine learning model is any one of the following:

6. the first time-varying image data is an M-mode image; The detection target is i) The diastolic and systolic positions of the annulus in a TAPSE (Tricuspid Annular Plane Systolic Excursion) measurement or a MAPSE (Mitoral Annular Plane Systolic Excursion) measurement, the timing of the diastolic and systolic periods on the time axis, or a trace of the annulus; or ii) In M-mode IVC (Inferior Vena Cava) diameter measurement, timing of expiration and inspiration, or tracing of the anterior and posterior walls of the inferior vena cava; The machine learning model according to claim 1, wherein the machine learning model is any one of the following:

7. 7. A program that causes a computer to realize an output function of outputting the detection target as an inference result from first predicted target time-varying image data obtained by normalizing second predicted target time-varying image data based on a received signal for image generation received by an ultrasound probe, using the machine learning model described in any one of claims 1 to 6.

8. The program according to claim 7 , wherein the detection target output as the inference result is information about a feature that is affected by a time change in a time-varying image.

9. The program according to claim 7, wherein the detection target output as the inference result is at least one of information regarding the position of one or more points in a time-varying image, information regarding width, information regarding waveform shape, information regarding time timing, information regarding a time interval, and information regarding a trace.

10. 8. The program according to claim 7, wherein the first prediction target time-varying image data is data obtained by normalizing the second prediction target time-varying image data in the time direction based on a sweep speed when generating the second prediction target time-varying image data.

11. 8. The program according to claim 7, wherein the first prediction target time-varying image data is time-varying image data cut out to a predetermined image width, or time-varying image data pasted onto background image data of a predetermined image width.

12. the second prediction target time-varying image data is a Doppler image, The detection target output as the inference result is i) A time section of a spectral waveform or a region of the spectral waveform that is the subject of VTI (Velocity-Time Integral) measurement; ii) Position or timing of PSV (Peak Systolic Velocity) and EDV (End Diastolic Velocity) on the time axis; iii) The boundary of the time section of the spectral waveform that is the target of blood flow measurement, or iv) The position or timing of the blood flow velocity of the E wave and the A wave on the time axis in the E / A measurement; 8. The program according to claim 7, wherein the program is any one of the above.

13. the second time-varying image data to be predicted is an M-mode image; The detection target is i) The diastolic and systolic positions of the annulus in a TAPSE (Tricuspid Annular Plane Systolic Excursion) measurement or a MAPSE (Mitoral Annular Plane Systolic Excursion) measurement, the timing of the diastolic and systolic periods on the time axis, or a trace of the annulus; or ii) In M-mode IVC (Inferior Vena Cava) diameter measurement, timing of expiration and inspiration, or tracing of the anterior and posterior walls of the inferior vena cava; 8. The program according to claim 7, wherein the program is any one of the above.

14. an ultrasonic probe for transmitting and receiving ultrasonic waves to and from a subject; an output unit that uses the machine learning model according to any one of claims 1 to 6 to output the detection target as an inference result from first predicted target time-varying image data obtained by normalizing second predicted target time-varying image data based on the reception signal received by the ultrasound probe in a time direction; An ultrasound diagnostic device having:

15. A database that stores the machine learning model according to any one of claims 1 to 6; an ultrasonic probe for transmitting and receiving ultrasonic waves to and from a subject; an output unit that uses the machine learning model to output the detection target as an inference result from first predicted target time-varying image data obtained by normalizing second predicted target time-varying image data based on the reception signal received by the ultrasound probe in the time direction; An ultrasound diagnostic system having:

16. an output unit that uses the machine learning model according to any one of claims 1 to 6 to output the detection target as an inference result from first predicted target time-varying image data obtained by normalizing second predicted target time-varying image data based on a reception signal received by an ultrasound probe in a time direction; An image processing device having:

17. At least one of training time-varying image data, which is a second time-varying image data obtained by normalizing the first time-varying image data based on the received signal for image generation received by the ultrasound probe in the time direction, and a third time-varying image data obtained by performing data extension to further transform the second time-varying image data in the time direction; Correct answer data including a detection target corresponding to the at least one training time-varying image data; A training device that trains a machine learning model using training data including pairs of normalizing the first time-varying image data in the time direction by processing the first time-varying image data based on a sweep speed when generating the first time-varying image data so that an image of the first time-varying image data, the image having a width direction in the time axis direction, is enlarged and / or reduced; The machine learning model comprises: A training device characterized by functioning to output a detection target as an inference result from first predicted target time-varying image data obtained by normalizing second predicted target time-varying image data based on a received signal for image generation received by an ultrasound probe in the time direction.

Images