Ultrasonic diagnostic device and data processing method

The knowledge distillation framework optimizes lightweight student models for ultrasound imaging by transferring knowledge from powerful teacher models, addressing computational and memory constraints, and improving image quality and frame rate in portable systems.

JP2024161915APending Publication Date: 2024-11-20CANON MEDICAL SYST CORP

Patent Information

Application Number
JP2024075952
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-07-28
Filing Date
2024-05-08
Publication Date
2024-11-20

AI Technical Summary

Technical Problem

Existing deep learning-based methods for harmonic ultrasound imaging require large computational and memory resources, making them unsuitable for real-time and portable ultrasound systems, and existing knowledge distillation models are not tailored for ultrasound imaging characteristics, leading to performance degradation and poor generalization.

Method used

A knowledge distillation framework that includes offline and online self-knowledge distillation methods to transfer knowledge from powerful teacher models to lightweight student models, optimizing them for ultrasound imaging by selecting the best performing teacher for each dataset and dynamically updating the student model during training.

Benefits of technology

The framework achieves robust, fast, and efficient ultrasound harmonic imaging with reduced artifacts and improved contrast-to-noise ratio, suitable for systems with limited computational power, enhancing image quality and frame rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024161915000001_ABST
    Figure 2024161915000001_ABST
Patent Text Reader

Abstract

To improve image quality.SOLUTION: An ultrasonic diagnostic device includes a processing circuit. The processing circuit inputs first training ultrasonic data including a fundamental wave component and a harmonic component to each of a plurality of teacher models, trains each teacher model of the plurality of teacher models with the first training ultrasonic data as teacher input data and with second training ultrasonic data including the harmonic component as teacher target data, acquires corresponding first estimation data output from the teacher model in response to the input of the first ultrasonic data to the teacher model for each teacher model of the plurality of teacher models, selects a first specific teacher model of the plurality of teacher models by evaluating the corresponding first estimation data output from each of the trained teacher models, and trains a student model with the first ultrasonic data as student input data and with the first estimation data as student target data.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The embodiments disclosed in this specification and the drawings relate to an ultrasound diagnostic apparatus and a data processing method. [Background technology]

[0002] Ultrasound imaging using higher harmonics can provide images with reduced artifacts, improved contrast-to-noise ratio, and improved lateral resolution. Recently, deep learning-based techniques have also enabled harmonic imaging with excellent image quality and fast image acquisition.

[0003] There have been many attempts in the past to develop lightweight models, but none of these approaches were specifically designed for harmonic ultrasound imaging with tailored ultrasound imaging characteristics. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] US Patent Application Publication No. 2023 / 0040181 Summary of the Invention [Problem to be solved by the invention]

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve image quality. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0006] An ultrasound diagnostic device according to an embodiment includes a processing circuit, which inputs first training ultrasound data including a fundamental wave component and a harmonic component to each of a plurality of teacher models, trains each of the plurality of teacher models using the first training ultrasound data as teacher input data and second training ultrasound data including the harmonic component as teacher target data, obtains corresponding first estimated data output from the teacher model in response to the input of the first ultrasound data to the teacher model for each of the plurality of teacher models, and selects a first specific teacher model from the plurality of teacher models by evaluating the corresponding first estimated data output from each of the trained teacher models, and trains a student model using the first ultrasound data as student input data and the first estimated data as student target data. [Brief description of the drawings]

[0007] [Figure 1A] FIG. 1A is a diagram showing an example of an exemplary conventional ultrasound image. [Figure 1B] FIG. 1B is a diagram illustrating an exemplary third harmonic. [Diagram 2] FIG. 2 is a system diagram for an ultrasound imaging system. [Diagram 3] FIG. 3 is a diagram illustrating a method for ultrasound tissue harmonic imaging. [Figure 4] FIG. 4 is a flow diagram of offline knowledge distillation. [Diagram 5] FIG. 5 is a flow diagram of online self-knowledge distillation. [Figure 6] FIG. 6 is a flowchart of a knowledge distillation training process according to an exemplary embodiment of the present disclosure. [Figure 7] FIG. 7 is a flowchart of a knowledge distillation training process according to an exemplary embodiment of the present disclosure. [Figure 8] FIG. 8 is a flow chart of an inference step according to an exemplary embodiment of the present disclosure. [Figure 9]FIG. 9 is a flowchart of a method for training a student model for ultrasound harmonic imaging according to an exemplary embodiment of the present disclosure. [Figure 10] FIG. 10 is a flow chart of the calculation steps of FIG. [Figure 11] FIG. 11 is a flow chart of the update steps of FIG. [Figure 12] FIG. 12 is a flow chart for an inference step, according to an exemplary embodiment of the present disclosure. [Figure 13] FIG. 13 is a block diagram illustrating an example computer system 204 for implementing machine learning training and inference methods according to example aspects of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, embodiments of an ultrasonic diagnostic apparatus and a data processing method will be described in detail with reference to the drawings.

[0009] In the drawings, like reference numbers indicate the same or corresponding parts throughout the several views.

[0010] By using higher harmonics in ultrasound harmonic imaging, images with significantly reduced artifacts, improved contrast-to-noise ratio, and improved lateral resolution can be obtained. For some anatomical structures, such as blood vessels, higher harmonic images with improved contrast are desired. For example, third harmonic imaging may provide images with fewer artifacts and improved resolution compared to second harmonic images. For example, FIG. 1A shows a conventional ultrasound image. In contrast, FIG. 1B shows an exemplary third harmonic image.

[0011] As ultrasound systems become more flexible and affordable in hospitals and clinics, it is desirable to have harmonic imaging with high image quality and accelerated imaging speed. Recently, deep learning based techniques have provided a good solution to improve harmonic imaging with excellent image quality and fast image acquisition. Deep learning can help improve harmonic image quality by sequentially transmitting fewer pulses with desired phases into the tissue along the same line and simultaneously improving the acquisition frame rate.

[0012] Deep learning based frameworks can directly use the second or third harmonics (i.e., IQ2 and IQ3), or alternatively, can use IQ data containing more fundamental frequencies, such as IQ0, IQ1, and IQ2, to obtain higher or complex harmonic images. Such frameworks can be feature-recognized, depth-dependent, and customized for patients with different BMIs and / or demographics.

[0013] However, one problem is that deep neural networks usually require deeper or wider network structures, which entail huge computational and memory costs. Thus, their large memory and numerical costs prevent deep neural networks from being applied to real-world solutions, especially for high frame rate real-time imaging. Reducing the model size without significantly compromising performance metrics is crucial for time- and memory-efficient ultrasound imaging. This is also particularly useful for portable ultrasound diagnostics, where memory size, inference speed, and network bandwidth are all severely constrained.

[0014] Many attempts have been made in the past in other fields to develop lightweight models, including the use of efficient architectures, model quantization, pruning, and knowledge distillation, but none of these approaches have been specifically designed for harmonic ultrasound imaging to align with ultrasound imaging characteristics.

[0015] For example, most of the current knowledge distillation models are designed for classification problems. In addition, the performance of the student model may be poor when the gap between the student model and the teacher model is large, and the student model may be suboptimal only in some cases and not for all datasets due to the diversity of the data. This is especially common in ultrasound imaging, where users can freely change the scanning depth, number of beams, focus, frequency, imaging anatomy, gain, etc. Therefore, it is desirable to have a more robust student model with better generalization ability for the task of ultrasound imaging.

[0016] 2 is a system diagram for an ultrasound imaging system. The ultrasound imaging system 200 can include any of a variety of transducers 202. Transducer types include convex, linear, and sector, as well as purpose-designed ones. Signals received by the transducer 202 are processed by a computer system 204. The ultrasound imaging system 200 can also include multi-harmonic compounding, where signals from individual beams are merged with overlapping data from adjacent beams.

[0017] Ultrasound images are created from sound waves at frequencies above the range of human hearing, typically around 1 to 10 MHz or higher. The transducer 202 emits high frequency waves and records the reflected waves (at the fundamental frequency) bouncing off tissue interfaces as a series of time-domain signals. One type of ultrasound image is a brightness image, also known as a B-mode image, which is a grayscale, intensity-based representation of the object being examined.

[0018] The raw signal received by the transducer 202 is in the high frequency domain and is known as Radio Frequency (RF) data. A series of signal processing steps are performed in the computer system 204 to convert the RF data into an ultrasound image, such as a B-mode image. One pre-processing step is to demodulate the RF data to baseband and decimate the signal to reduce the bandwidth required to store the data. This new signal is called an In-phase and Quadrature phase (IQ) signal, and is typically represented as a complex number. In this disclosure, IQ data and RF data are used interchangeably since they both represent raw data from the transducer but in different formats. Additionally, the deep neural network embodiments of the present disclosure are configured to take as input either IQ data or an ultrasound image based on IQ data.

[0019] The computer system 204 can be embedded in a portable ultrasound diagnostic device, can be a remote server, or can be a cloud service accessed via the Internet. The ultrasound imaging system 200 includes at least one display device 206 for displaying one or more ultrasound images. The display device 206 can be either an LCD display, an LED display, an organic LED display, etc. The display size and resolution are sufficient to display the ultrasound images output by the computer system 204.

[0020] The above-described computer system 204 may perform the following processing methods.

[0021] In one embodiment, the best-trained teacher model among multiple teacher models is used for knowledge distillation, as described in more detail below. This idea is implemented for offline knowledge distillation, and can alternatively be used for online self-knowledge distillation.

[0022] In one embodiment, in offline knowledge distillation, the method first trains a set of teacher models using heavy and powerful deep networks on a large dataset. Convolutional Neural Networks (CNNs) have been used in visual recognition tasks. CNNs can be trained on a large set of training images, such as a million training images, for example, as in the ImageNet dataset. ImageNet had eight layers and millions of parameters. Very heavy and powerful deep convolutional networks can be used for large-scale image recognition. The deep neural network can be, for example, any deep neural network for vision. As mentioned above, convolutional neural networks have shown excellent results, and the U-Net architecture in particular allows training on smaller datasets than a full convolutional neural network. Other types of deep neural networks, including multi-layer perceptrons or vision transformers, may also be used to implement the deep neural network.

[0023] Then, among the results of the teacher model, for a given training data set, the knowledge of the best teacher is transferred to a student model with a smaller network, e.g., fewer layers and parameters. Only the results of the best teacher model for a given input are used to train the student model. The best teacher model is, for example, the network with the highest accuracy, but other criteria can be used. Thus, the student can learn from multiple teachers, ensuring that it always learns from the best known teacher for a set of different training data. This strategy also allows for the generation of more training data pairs for training the student model, for example, when ground truth target data is missing. This allows for effective data augmentation to increase the diversity of the data (e.g., various imaging conditions, frequencies, imaging depths, number of beams, etc.).

[0024] In one embodiment, in online self-knowledge distillation, a student network is trained incrementally by distilling its own knowledge without a pre-trained "teacher" network. To use the concept of the best teacher, the student model is updated using the best past student model to guide the training process of the current model. This strategy uses a simple network structure and improves the generalization ability of the model. This is very useful for reducing the gap between the student model and the "teacher" model, making the network more robust in various imaging conditions, yet lightweight in structure.

[0025] FIG. 3 illustrates a method of ultrasound tissue harmonic imaging. To obtain more signals in the ultrasound far-field image, both second and third harmonic images can be acquired. As mentioned above, one method is the DTHI method, in which two pulses are transmitted simultaneously at different frequencies, called f1 and f2. In particular, in addition to their second harmonic frequencies (2f1 and 2f2), the sum and difference of the transmitted frequencies (f2+f1 and f2-f1, respectively) are generated in the tissue. The second harmonic signals 316 at the lower frequency (2f1) and difference frequency (f2-f1), respectively, are detected by the transducer. However, this method requires a long time interval 312 for pulse generation. This results in a slow frame rate, which leads to poor quality imaging, especially when there is tissue motion.

[0026] FIG. 4 is a flow diagram of one embodiment of offline knowledge distillation.

[0027] In step S402, multiple heavy and powerful deep teacher models 452 are trained using various IQ input data 462 and output data 464, which are desired harmonic IQ data / images. In one embodiment, a deep learning-based framework for each teacher model is configured to input various different combinations of IQ data received by the transducer 202 and subjected to data processing steps in the computer system 204, and output desired images with improved image quality, fewer near-field artifacts, improved contrast, and deeper penetration. The deep learning-based framework can directly use second or third harmonics, or alternatively, can use data including fundamental frequencies. The input IQ data can include a combination of fundamental frequency signals, second harmonic signals, and third harmonic signals (IQ0). The IQ data can include a combination of fundamental ultrasound frequency and third harmonic (IQ1). The input IQ data can also include only second harmonic signals (IQ2) or only third harmonic signals (IQ3). The input IQ data can also be other higher order harmonics greater than third.

[0028] In step S406, data augmentation can be performed as needed, and each of the trained teacher models 452 runs inference on various test datasets. For each dataset, the teacher model with the best output results / feature map is selected as the teacher model 472 used to train a lightweight student model 474 for that dataset.

[0029] In step S410, a student model 474 is trained for harmonic imaging using the augmented data pair (IQ input data 462 and knowledge distillation data 468 resulting from the output data 464 of the best teacher model for the corresponding IQ input data 462).

[0030] The offline knowledge distillation process is further illustrated in the flowcharts of Figures 6 to 8. In Figure 6, the training process includes, in step S602, inputting first training ultrasound data including fundamental and harmonic components into each of a plurality of teacher models, and in step S604, training each teacher model of the plurality of teacher models using the first training ultrasound data as teacher input data and the second training ultrasound data including harmonic components as teacher target data.

[0031] In step S606, the method includes, for each teacher model of the plurality of trained teacher models, obtaining corresponding first estimated data output from the teacher model in response to input of the first ultrasound data to the teacher model.

[0032] In step S608, the method includes selecting a first particular teacher model from the plurality of trained teacher models by evaluating corresponding first estimation data output from each of the trained teacher models.

[0033] In step S610, a student model is trained using the first ultrasound data as student input data and the corresponding first estimation data of the selected first specific teacher model as student target data.

[0034] Figure 7 is a flow chart of further steps of the method of Figure 6. In step S702, the method includes, for each teacher model of the trained plurality of teacher models, obtaining corresponding second estimated data output from the teacher model in response to inputting second ultrasound data different from the first ultrasound data to the teacher model.

[0035] In step S704, the method includes selecting a second particular teacher model from the plurality of trained teacher models by evaluating corresponding second estimation data output from each of the trained teacher models.

[0036] In step S706, the method includes training a student model using the second ultrasound data as student input data and the corresponding second estimated data of the selected second particular teacher model as student target data.

[0037] Figure 8 is a flow chart of further steps of Figure 6. In step S802, the method includes inputting input ultrasound data into a trained student model to generate output ultrasound data including harmonic content.

[0038] FIG. 5 is a flow diagram of one embodiment of an online self-knowledge distillation training process. In step S402, the IQ input data 462 can be pre-processed and then input to the student model 474. The target IQ data corresponding to the IQ input data 462 is the output data 464 shown in FIG. 5. In step S404, the same IQ input data 462 input to the student model is input to a pseudo best teacher model 482, which may be the previous best student model. Here, the student model 474 and the pseudo best teacher model 482 can have, for example, the same network structure and number of parameters. The knowledge distillation data 468, which is the output of the student model, the output of the pseudo teacher model 484, and the desired / target output data 464 are used to calculate a loss function value, which is then used to update the parameters of the student model 474. During the training of the student model 474, the parameters of the pseudo best teacher model remain fixed.

[0039] In one embodiment, the online training process can be represented by the following pseudocode: S ,θ S ) represents the student model 474, and θ s is the set of trainable parameters, (F T ,θ T ) represents the pseudo best teacher model482, and θ T is the set of frozen parameters, and L(Z S , y';θ S) is the ground truth y' (desired IQ output data 464) for the model input x' and the student model output Z S (Knowledge Distillation Data 468) is the loss between L(Z S , Z T ;θ S ) is the pseudo best teacher model output Z T (Output 484) and student model output Z S (knowledge distillation data 468), and λ is the loss weight.

[0040]

number

[0041] Here, the weights in the loss function, λ, can be a constant hyperparameter, or we can set λ as a function of epochs, e.g., stepwise, exponential, or linear growth. Furthermore, we can utilize different loss functions to compute the weighted loss, such as MSE, MAE, feature map-based, or loss functions that consider outlier detection on targets learned from noisy teachers.

[0042] The online knowledge distillation process is further illustrated in the flowcharts of Figures 9 to 12. Figure 9 is a flowchart of a method for training a student model for ultrasound harmonic imaging.

[0043] The training process includes, in step S902, acquiring first ultrasound data including a fundamental component and a harmonic component as input ultrasound data, and acquiring second ultrasound data including a harmonic component as target output ultrasound data corresponding to the first ultrasound data.

[0044] In step S904, the method includes inputting the first ultrasound data into a previously trained teacher model to generate teacher output ultrasound data.

[0045] In step S906, the method includes inputting the first ultrasound data into a student model to generate student output ultrasound data.

[0046] In step S908, the method includes calculating a loss value of a loss function based on the generated teacher output ultrasound data, the generated student output ultrasound data, and the target output ultrasound data.

[0047] In step S910, the method includes updating parameters of the student model based on the calculated loss value.

[0048] In step S912, the method repeats the steps of acquiring, inputting, calculating, and updating for different input ultrasound data and corresponding different second ultrasound data until the parameters of the student model meet a convergence criterion.

[0049] FIG. 10 is a flowchart of the step of calculating the loss value of the loss function in FIG. 9 (ie, step S908).

[0050] In step S1002, the method includes calculating a first loss value of a first loss function based on the generated student output ultrasound data and the target output ultrasound data.

[0051] In step S1004, the method includes calculating a second loss value of a second loss function based on the generated teacher-output ultrasound data and the generated student-output ultrasound data.

[0052] In step S1006, the method includes calculating a loss value as a weighted sum of the first loss value and the second loss value.

[0053] Figure 11 is a flow chart of the updating step (i.e., step S910) of Figure 9. Step S1102 involves updating parameters of the student model without updating parameters of the trained teacher model.

[0054] 12 is a flow chart of inference steps according to an exemplary embodiment of the present disclosure. In step S1202, the method includes inputting input ultrasound data into a trained student model to generate output ultrasound data including harmonic content.

[0055] In this disclosure, there are various methods of performing knowledge distillation for harmonic imaging based on different inputs and required outputs, including the following:

[0056] 1. Directly use the inverse pulse sequence to obtain IQ1 (fundamental + third harmonic), and use deep learning techniques to extract the third harmonic to replace the second harmonic imaging used in clinical practice. Then, use a teacher-student model to train a compact network for high-speed imaging.

[0057] 2. Directly use the second and third harmonic IQ data or images to train a student network to obtain the desired target harmonic IQ data / images, including end-to-end training, and training a fusion map, and calculate the loss based on the fusion map.

[0058] 3. Directly use IQ1 (fundamental + 3rd harmonic) and IQ2 (2nd harmonic) data or images to train a student model to obtain the target IQ data / image, including end-to-end training, and training the network to obtain IQ3 from IQ1, and smartly fuse the estimated IQ3 with IQ2.

[0059] 4. Directly use IQ0 (fundamental + 2nd harmonic + 3rd harmonic) to obtain the target harmonic IQ data / image, including end-to-end training, and training the network to split IQ2 (2nd harmonic) and IQ3 (3rd harmonic) from IQ0, and output the desired IQ data / image.

[0060] 5. For higher harmonics, the framework can be extended to obtain higher harmonic IQ data / images.

[0061] The disclosed method aims to perform accelerated and enhanced ultrasound tissue harmonic imaging using knowledge distillation of deep convolutional neural networks. Previous work on knowledge distillation has mostly targeted traditional computer vision classification problems and is not directly suitable for ultrasound imaging and regression problems.

[0062] Both offline knowledge distillation as the best teacher model and online self-knowledge distillation for the best teacher are provided to train lightweight student models for harmonic imaging. In offline training, the best performing teacher among multiple teacher models used for student training is guaranteed regardless of the diversity of the data. In online training, the teacher network is not a fixed model with weights, but dynamically evolves as training progresses. Because it shares the same structure as the student, time is saved in online training compared to using a teacher network with a different structure.

[0063] As a result, the framework provides the best performing teacher to minimize the gap between teacher and student, and also improves the generalization ability of the lightweight student model. This is particularly useful for generating robust and fast DCNN networks for ultrasound harmonic imaging under various scanning conditions and on various systems, even those with limited computational power and bandwidth.

[0064] Depending on the input and task requirements, the method can provide deeper reaching harmonics with fewer artifacts, improved contrast, and accelerated frame rates. The method can provide fast feature-aware depth-dependent harmonics, which do not necessarily have to be pure harmonics or pure complex harmonics, but can also be extended harmonics (depth-dependent, feature-aware) with greatly reduced frame rates.

[0065] In ultrasound harmonic imaging, to acquire higher harmonics, traditional multiple pulse sequence methods are frame-rate limited and suffer from motion artifacts. Deep learning based methods can provide a solution to obtain feature-aware superior harmonic image quality. However, direct implementation of deep networks may be unusable due to the enormous computational power and memory required. This knowledge distillation framework provides a robust solution to solve advanced ultrasound harmonic imaging problems at accelerated imaging speeds. The knowledge distillation framework for high-speed ultrasound harmonic imaging improves harmonic image quality while enhancing frame rate.

[0066] In this disclosure, a trained student network is used to obtain higher harmonics or enhanced harmonics with a deep network that transmits fewer pulse sequences. A best-of-breed teacher training framework, both offline and online, is provided for knowledge distillation in ultrasound harmonic imaging. The model accelerates harmonic imaging and can be easily implemented in ultrasound systems with limited computational power and memory. The model architecture is particularly suitable for low-cost ultrasound systems, including portable ultrasound systems, to perform high-quality harmonic imaging.

[0067] FIG. 13 is a block diagram illustrating an exemplary computer system 204 for implementing the machine learning training and inference method according to an exemplary aspect of the present disclosure. The computer system may be an AI workstation running a server operating system, such as, for example, Ubuntu Linux OS, Windows Server, a version of Unix OS, or Mac OS Server. The computer system 1300 may include one or more central processing units (CPUs) 1350 having multiple cores. The computer system 1300 may include a graphics board 1312 having multiple GPUs, each GPU having GPU memory. The graphics board 1312 may perform many of the mathematical operations of the disclosed machine learning method. In other embodiments, the computer system 1300 may include a machine learning engine. The machine learning engine may perform many of the mathematical operations of the disclosed machine learning method. The computer system 1300 includes a main memory 1302, typically a random access memory RAM, including software executed by the CPU 1350 and the GPU (graphics board 1312), and a non-volatile storage device 1304 for storing data and software programs. Several interfaces may be provided for interacting with the computer system 1300, including an I / O bus interface 1310, input / peripherals 1318 such as a keyboard, touchpad, mouse, etc., a display adapter 1316 and one or more displays 1308 (206), and a network controller 1306 that enables wired or wireless communication over a network 99. The interfaces, memory, and processor may communicate via a system bus 1326. The computer system 1300 includes a power supply 1321, which may be a redundant power supply.

[0068] In some embodiments, the computer system 1300 may include a server CPU and a graphics card from NVIDIA®, whose GPU has multiple CUDA® cores.

[0069] The above hardware description is a non-limiting example of a corresponding structure for performing the functions described herein. Also, the embodiment is not limited to an ultrasound diagnostic device, and can be similarly applied to other medical modalities. Also, an image processing device independent of the ultrasound diagnostic device may perform the same processing.

[0070] Numerous modifications and variations of the present disclosure are possible in light of the above teachings, and it is therefore to be understood that, within the scope of the appended claims, the invention may be practiced otherwise than as specifically described herein.

[0071] According to at least one of the embodiments described above, image quality can be improved.

[0072] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0073] 452 Teacher Model 462 IQ input data 464 Output Data 468 Knowledge Distillation Data 472 Teacher Model 474 Student Model 482 Pseudo-Teacher Model 484 Output

Claims

1. inputting first training ultrasound data including a fundamental wave component and a harmonic component into each of a plurality of teacher models, and training each teacher model of the plurality of teacher models using the first training ultrasound data as teacher input data and the second training ultrasound data including the harmonic component as teacher target data; For each teacher model of the plurality of teacher models, in response to inputting first ultrasound data to the teacher model, obtaining corresponding first estimated data output from the teacher model; a processing circuit for selecting a first particular teacher model from the plurality of teacher models by evaluating the corresponding first estimated data output from each of the trained teacher models, and training a student model using the first ultrasound data as student input data and the first estimated data as student target data. An ultrasound diagnostic device comprising:

2. The ultrasound diagnostic apparatus of claim 1 , wherein each of the plurality of trained teacher models has a different model structure from the student model and more parameters than the student model.

3. The processing circuitry comprises: For each teacher model of the plurality of teacher models, in response to input of second ultrasound data different from the first ultrasound data to the teacher model, corresponding second estimated data output from the teacher model is obtained; selecting a second particular teacher model from the plurality of teacher models by evaluating the second estimation data; The ultrasound diagnostic apparatus of claim 1 , further comprising: training the student model using the second ultrasound data as the student input data and the corresponding second estimated data of the selected second specific teacher model as the student target data.

4. The ultrasound diagnostic device of claim 1 , wherein the processing circuitry selects the second particular teacher model that is different from the selected first particular teacher model.

5. The ultrasound diagnostic device of claim 1 , wherein the processing circuitry generates output ultrasound data including the harmonic components by inputting input ultrasound data to the student model.

6. Acquire first ultrasonic data including a fundamental wave component and a harmonic component as input ultrasonic data, and acquire second ultrasonic data including the harmonic component as target output ultrasonic data corresponding to the first ultrasonic data; inputting the first ultrasound data into a previously trained teacher model to generate teacher output ultrasound data; inputting the first ultrasound data into a student model to generate student output ultrasound data; Calculate a loss value of a loss function based on the teacher output ultrasound data, the student output ultrasound data, and the target output ultrasound data; a processing circuit for updating parameters of the student model based on the loss value; An ultrasound diagnostic device comprising:

7. In calculating the loss value, the processing circuitry Calculating a first loss value of a first loss function based on the student output ultrasound data and the target output ultrasound data; Calculating a second loss value of a second loss function based on the teacher output ultrasound data and the student output ultrasound data; The ultrasound diagnostic apparatus of claim 6 , wherein the loss value is calculated as a weighted sum of the first loss value and the second loss value.

8. 7. The ultrasound diagnostic device of claim 6, wherein the processing circuitry is further configured to repeat the acquiring, inputting, calculating, and updating steps for different input ultrasound data and corresponding different second ultrasound data until the parameters of the student model meet a convergence criterion.

9. The ultrasound diagnostic apparatus of claim 6 , wherein, in the updating, the processing circuit updates the parameters of the student model without updating parameters of the teacher model.

10. The ultrasound diagnostic device of claim 6 , wherein the processing circuitry generates output ultrasound data including the harmonic components by inputting input ultrasound data to the student model.

11. inputting first training ultrasound data including a fundamental wave component and a harmonic component into each of a plurality of teacher models, and training each teacher model of the plurality of teacher models using the first training ultrasound as teacher input data and the second training ultrasound data including the harmonic component as teacher target data; For each teacher model of the plurality of teacher models, in response to inputting first ultrasound data to the teacher model, obtaining corresponding first estimated data output from the teacher model; selecting a first particular teacher model from the plurality of teacher models by evaluating the first estimation data; A data processing method comprising: training a student model using the first ultrasound data as student input data and the first estimated data as student target data.

12. Acquire first ultrasonic data including a fundamental wave component and a harmonic component as input ultrasonic data, and acquire second ultrasonic data including the harmonic component as target output ultrasonic data corresponding to the first ultrasonic data; inputting the first ultrasound data into a previously trained teacher model to generate teacher output ultrasound data; inputting the first ultrasound data into a student model to generate student output ultrasound data; Calculate a loss value of a loss function based on the teacher output ultrasound data, the student output ultrasound data, and the target output ultrasound data; updating parameters of the student model based on the loss value; (c) a method for processing data,

Citation Information

Patent Citations

  • System and method for improving named entity recognition

    US20230040181A1

Cited By

  • In-vehicle device, connection destination notification method, and computer program product

    CN121711662A