Ultrasound system for automatically and dynamically setting imaging parameters based on organ detection
By integrating machine learning models in an ultrasound system to identify biological tissue types in real time and automatically adjust imaging parameters, the problem of error-prone users' manual adjustment of parameters is solved, and image quality and inspection efficiency are improved.
Patent Information
- Application Number
- CN201980049737.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-07-26
- Filing Date
- 2019-07-19
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2039-07-19
AI Technical Summary
Existing ultrasound imaging systems require users to manually adjust imaging parameters when inspecting different organs, which can easily lead to incorrect selection, resulting in poor image quality and quantitative measurement errors, especially in emergencies, time is tight and inconvenient to switch presets.
By integrating processors in ultrasound systems, using machine learning models (such as neural networks) to identify biological tissue types in real time, and automatically adjust imaging parameters, selecting appropriate preset settings to optimize image quality.
It realizes automatic optimization of imaging parameters without manual adjustment by users, improves image quality and quantitative measurement accuracy, and reduces inspection time, especially in emergency situations.
Smart Images

Figure CN112513674B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to ultrasound imaging systems and methods for ultrasonically examining biological tissue, and more particularly to systems configured to automatically adjust imaging parameters of the system to tissue-specific settings based on organ detection. Background Art
[0002] Ultrasound imaging is commonly used to non-invasively image a patient's internal tissues or organs, such as for diagnosing any number of different diseases or monitoring their progression or the success of treatment. When performing an ultrasound examination, a user (e.g., a sonographer or clinician) may need to frequently adjust imaging parameter settings (e.g., depth, focus, frequency, gain, TGC, imaging mode, etc.) to obtain high quality images.
[0003] In a typical system, before starting any ultrasound examination, the user is required to select a preset that sets one or more imaging parameters of the system to settings that are generally optimized for the specific organ / tissue being studied. Therefore, a tissue-specific preset (TSP) defines the settings of one or more of the imaging parameters in the system, which may be suitable for a specific imaging application. Modern imaging systems provide a user interface for switching between different TSPs. A user selection screen may be presented to the user, for example, based on the transducer application type, for the user to select a TSP that the user deems appropriate. Due to the presence of a variety of TSPs in a typical imaging system, the user can accidentally select a preset that is not the most suitable for a given patient. For example, this may occur when a clinician performs an examination of multiple organs of a patient during the same scanning session, and the multiple organs may be better imaged under different settings. Moreover, in an emergency setting, the user usually does not have time to preselect a suitable TSP, and may ultimately perform the examination with the wrong parameters, resulting in suboptimal ultrasound images and incorrect quantitative measurements. Therefore, designers and manufacturers of ultrasound imaging systems continue to seek improvements thereto. Summary of the invention
[0004] The present disclosure relates to ultrasound imaging systems and methods for ultrasonically examining biological tissue, and more particularly to systems configured to automatically adjust imaging parameters of the system to tissue-specific settings based on organ detection.
[0005] According to some examples of the present disclosure, an ultrasound system may include: a probe configured to emit ultrasound toward an object to generate an ultrasound image of biological tissue of the object; and a processor configured to generate and enable the ultrasound imaging system to display a live ultrasound image stream of the biological tissue in real time based on multiple imaging parameters of the ultrasound system.
[0006] The processor may also be configured to: receive an ultrasound image from a live ultrasound image stream in real time, receive an identification of a type of biological tissue in the ultrasound image based on the type of biological tissue and in some cases one or more additional input parameters, generate at least one predicted setting for at least one of a plurality of imaging parameters, and automatically apply the at least one predicted setting to the corresponding imaging parameter for subsequent live imaging. In some embodiments, the processor may employ a neural network to generate (one or more) predicted settings. In some embodiments, the identification of the biological tissue is performed by the processor, for example using a machine learning model, such as a suitably trained machine learning organ classification model. In some examples, the system may optionally further include a memory storing a plurality of presets, each preset defining one or more settings for at least one of the imaging parameters of the ultrasound imaging system, and the processor may be configured to select one of the plurality of stored presets based on the type of biological tissue, and automatically apply the selected preset, for example before or while generating at least one predicted setting, to adjust one or more of the plurality of imaging parameters of the system to the settings defined by the selected preset. In some examples, the predicted settings may be generated using the imaging settings defined by the selected preset as input, for example, by a machine learning model suitably trained to customize the preset settings according to patient-specific and / or user-specific preferences.
[0007] In some examples, the system can utilize a suitably trained machine learning regression model to generate (one or more) patient-specific and / or user-specific settings, also referred to herein as (one or more) predicted settings. In some examples, the regression model can be trained using data extracted from system logs from multiple ultrasound imaging systems. In some examples, the artificial neural network used by the ultrasound system can be configured to output at least one predicted setting in response to inputs including the type of biological tissue, user identification information, patient identification information, and corresponding settings defined by a selected preset. In some examples, the artificial neural network can include multiple layers, including a first input layer configured to receive an input of size n+1 and an output layer configured to generate an output of size n, and wherein n is equal to the number of settings defined by the selected preset.
[0008] According to some examples, a method for ultrasonically examining biological tissue may include: receiving, by a processor of an ultrasound system, an ultrasound image from a live ultrasound image stream in real time, receiving an identification of a type of biological tissue in the ultrasound image, selecting one of a plurality of presets stored in a memory of the ultrasound system based on the type of biological tissue in the ultrasound image, automatically adjusting one or more imaging parameters of the ultrasound system to settings defined by the selected preset based on the type of biological tissue, identifying one or more user-specific settings, patient-specific settings, or a combination thereof for at least one of the one or more imaging parameters in the imaging parameters using an artificial neural network, and automatically adjusting at least one imaging parameter according to the one or more user-specific settings, patient-specific settings, or a combination thereof for subsequent live imaging.
[0009] Any of the methods described herein, or steps thereof, may be embodied in a non-transitory computer-readable medium comprising executable instructions which, when executed, may cause a processor of a medical imaging system to perform the methods or steps embodied herein. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 A flow chart showing a process for ultrasonically examining biological tissue according to some examples of the present disclosure is shown.
[0011] Figure 2 is a block diagram of a system in accordance with the principles of the present disclosure.
[0012] Figure 3 is a block diagram of a processor for an ultrasound imaging system according to the present disclosure.
[0013] Figure 4 A method for training and deploying an artificial neural network (or machine learning model) in an ultrasound imaging system according to the present disclosure is shown.
[0014] Figure 5 A diagram of an example neural network architecture according to the present disclosure is shown, which can be used to implement an anatomical structure classification model.
[0015] Figure 6 A diagram is shown of an example neural network architecture according to the present disclosure that can be used to implement a set prediction model.
[0016] Fig. 7A and 7B A portion of an ultrasound system log is shown that captures setting adjustments made during an ultrasound scan, which may be used to compile training data for a setting prediction model according to the present disclosure.
[0017] Figure 8Graphs showing data extracted from a system log in accordance with the present disclosure, and more particularly showing changes in three exemplary imaging parameters plotted as a function of time.
[0018] Fig. 9 A table showing input parameters for use in training a prediction model according to the present disclosure is shown.
[0019] Fig. 10A , 10B 10C show screen captures of a graphical user interface for manually selecting a preset on an example ultrasound system according to some examples herein.
[0020] Fig.11A and 11B Ultrasound images of two different types of tissue are shown, along with corresponding imaging settings suitable for imaging the illustrated types of tissue.
[0021] Fig. 12A , 12B 12C show examples of ultrasound images illustrating the effects of the use of correct or incorrect imaging parameters on image quality. DETAILED DESCRIPTION
[0022] The following description of certain embodiments is merely exemplary in nature and is in no way intended to limit the invention or its application or use. In the following detailed description of the embodiments of the present system and method, reference is made to the accompanying drawings, which form part of this document and show by way of illustration specific embodiments in which the described systems and methods can be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the currently disclosed systems and methods, and it should be understood that other embodiments may be utilized and that structural and logical changes may be made without departing from the spirit and scope of the present system. In addition, for the purpose of clarity, when the detailed description of certain features will be obvious to those skilled in the art, they will not be discussed so as not to obscure the description of the present system. Therefore, the following detailed description should not be regarded as having a limiting meaning, and the scope of the present system is limited only by the appended claims.
[0023] Ultrasound imaging systems include a number of reconfigurable imaging parameters that control, for example, brightness / contrast (gain) on the image, depth of ultrasound beam focus, imaging frequency, time gain compensation (TCG), user modes (i.e., fundamental frequency imaging, tissue harmonic imaging, and compound imaging), and flow velocity range for color Doppler quantification. Other imaging parameters may include focus (or focal area), line density, persistence, dynamic range of display (or compression), etc. Different organs and structures of the human body can require very different imaging parameters, and the correct choice of imaging parameters has a great impact on the readability and interpretation of the output image for the end user (e.g., a radiologist). For example, if Fig. 10A -C, for a given imaging application (e.g., abdomen, breast, and cardiac), on a typical modern ultrasound system (in this example, on an ultrasound imaging system sold by PHILIPS under the trade name EPIQ), a variety of different presets may be available for a given transducer type. Fig. 10A A user interface is shown for selecting between available presets for abdominal imaging, which is often performed using a PHILIPS C5-1 curvilinear probe. As illustrated, six different presets 1010 are available for abdominal imaging, six presets 1012 are available for obstetric imaging, two presets 1014 are available for gynecological imaging, and one preset 1016 is used for pediatric abdominal examinations, as well as additional optional presets 1018 (e.g., WHC TDI, intervention, etc.) for other imaging applications. Fig. 10B A user interface is shown for selecting between available presets for superficial or breast imaging, which are often performed using a PHILIPS L12-5 linear probe. As illustrated, a variety of different presets 1022 are available, including breast, thyroid, testicular, superficial, musculoskeletal (MSK) general and superficial, abdominal bowel, pediatric abdomen, obstetric general, and various presets for vascular applications (e.g., carotid, venous, arterial, superficial). Fig. 10C A selection interface is shown for presets associated with a PHILIPS S5-1 sector array probe, which can be used for cardiac imaging and, therefore, can be associated on this exemplary system with presets 1024 for echocardiography applications (e.g., adult echo, echo pen) and presets 1026 for other imaging applications (e.g., abdominal general, adult general, pediatric, etc.).
[0024] Imaging parameter settings can control both qualitative aspects of imaging (e.g., dynamic range of display, depth, imaging frequency, etc.) and quantitative aspects of imaging (e.g., flow rate), and the ideal qualitative and quantitative imaging parameters can vary from organ to organ. Fig.11A and 11B As shown, with abdominal renal preset ( Fig.11A 1110) and OB Cardiac presets ( Fig. 11BThe blood flow velocities associated with the Doppler images (see ranges 1112 and 1114) are very different. More specifically, for abdominal renal imaging, the blood flow range 1112 can be about ±18 cm / s, while in OB fetal heart examinations, the blood flow range 1114 can be much higher, up to about ±38.5 cm / s. When performing quantitative measurements, the range of blood flow velocities is an important parameter that needs to be set accurately in order to avoid aliasing artifacts in the Doppler image. Aliasing occurs when the peak velocity of the blood flow is faster than the Doppler shift set on the Doppler scale. Therefore, selecting an appropriate preset with the correct blood flow range ensures that an appropriate pulse sequence with an appropriate pulse repetition frequency (sampling rate) will be used during imaging. Conversely, if an incorrect preset is used and an aliased signal is recorded, the peak flow velocity will be truncated in the aliased signal, introducing errors and potentially leading to misdiagnosis. Fig. 12A -C shows a further example of a cardiac image illustrating the effect of incorrect imaging parameters of the output image. Fig. 12A A relatively poor quality image is shown, showing blurred heart wall, low image contrast and strong clutter artifacts due to the incorrect use of abdominal TSP. Fig. 12B The improved image quality by selecting the correct preset and thus applying appropriate imaging parameter settings is shown. Fig. 12C It is illustrated that even for a given organ-specific application, appropriately selected presets may be applied, and organ-specific settings may be improved by additional fine-tuning of the settings of the preset application for further improvement of image quality, which may be achieved using an adaptive TSP model according to the present disclosure.
[0025] The TSPs available on ultrasound scanners use population-based settings provided by the manufacturer. Once a selection is made by the user, they very often have to keep adjusting the parameters to match the acoustic properties of each patient. Due to the presence of a wide variety of TSPs in imaging systems, users can inadvertently use presets during an examination that are not ideal or optimal for a particular imaging application. For example, in an emergency care setting where time may be critical, clinicians may not have time to switch TSPs when examining different organs. In addition, users often customize the pre-programmed imaging parameters for each patient according to their own preferences when interpreting and reading ultrasound images. Different users may select very different settings for the same patient, and sometimes users may save customized system settings as their dedicated TSPs. In some cases, these customized or user-specific TSPs may be preferred by a given user over the pre-programmed (factory default) TSPs.
[0026] As described above, many types of transducers can be used for use in ultrasound examinations. For example, different types of transducers can be designed to image different organs of the body and / or for specific clinical applications. In some cases, the same transducer can be used to image different organs and can be equipped with several examination-specific imaging presets. In the daily practice of ultrasound imaging, the ultrasound examiner must make a pre-selection of the initial imaging settings to start the ultrasound examination, so TSP selection is usually required before starting any examination. The user usually has to browse the entire list of available presets and then select one for the current examination. In addition, since the available TSPs are based on a general population (e.g., the factory default presets are usually optimized for a wide variety of patients), the ultrasound examiner may have to use various buttons and knobs on the scanner to adjust the settings of the selected preset to find the right combination of imaging parameters for the current patient. These processes can be time-consuming and are particularly undesirable in emergency situations, where time is usually critical and this type of reconfiguration of the system during the examination takes up valuable time that should otherwise be devoted to caring for the patient.
[0027] An ultrasound imaging system according to the present disclosure may be configured to detect an incorrectly selected preset and automatically apply a correct preset, which is a preset determined by the system to be the best preset for the current scan (e.g., best suited to the current probe, imaging application or organ, patient, user, etc.). Automatic selection of the correct preset may be accomplished by first identifying the type of organ being imaged and then retrieving the appropriate preset for that organ. This identification and preset selection may be performed periodically during an examination, e.g., continuously while scanning a patient, so that if the sonographer moves to image another organ without switching presets, the system will automatically detect the change and apply the appropriate preset.
[0028] In some examples, the system may also be configured to adjust or modulate the settings of the system's imaging parameters (e.g., settings applied by a given TSP) to settings determined by the system to be more appropriate for the particular patient being examined and / or settings determined by the system to be more appropriate for the current operator of the system (e.g., the sonographer performing the examination). To this end, the system may be configured to identify user-specific settings (e.g., retrieving and isolating customized settings associated with a particular current user) and patient-specific settings (e.g., settings previously used for the same patient and the same organ) from a data store containing previous settings, and automatically adjust the current settings to the retrieved user-specific and / or patient-specific settings. The term organ detection or identification may imply identification of a particular organ being examined (e.g., heart or liver) or more generally a type of tissue (e.g., kidney, liver, blood vessel, breast tissue, etc.).
[0029] Figure 1A flow chart of an example process 100 for automatically reconfiguring (e.g., adjusting imaging parameter settings) an ultrasound imaging system (e.g., ultrasound scanner 101-j) during live imaging in accordance with the principles of the present disclosure is shown. The ultrasound scanner 101-j may be configured to acquire and generate live images of biological tissue from ultrasound emitted toward the biological tissue. Figure 1 As shown, initially, images may be acquired using imaging settings that may not be ideal for a given imaging application. As described, in addition to performing conventional signal and image processing of the echo signals (e.g., for the purpose of generating an image), the system may also couple one or more of the live images to the processor in parallel to identify the type of tissue being imaged. The identification of the type of tissue may be used by the system to automatically select and apply tissue-specific imaging parameters, thereby avoiding the need for an operator to manually switch between presets when moving to a new area of the anatomy. Process 100 may begin at box 110 by receiving an ultrasound image in a processor that implements a tissue identification module (also referred to as a detection module) configured to identify the type of tissue being imaged. The tissue identification module may be implemented using at least one appropriately trained artificial neural network (or machine learning model). The processor of the ultrasound scanner 101-j may use a machine learning model having any suitable architecture (e.g., such as Figure 1 ) which is trained for the task of recognizing the type of tissue represented in an input ultrasound image. The machine learning model may reside on the ultrasound scanner 101-j or on a remote computing device communicatively coupled to the ultrasound scanner 101-j.
[0030] At box 112, the process continues by identifying the type of tissue being imaged and selecting a tissue-specific preset for the type of tissue identified. As described, the identification can be performed by a processor resident on the ultrasound scanner 101-j, which implements (i.e., executes processor executable instructions) a machine learning classification model. In some examples, the identification can be performed by a remote (e.g., network-based or cloud-based) processor, and the identification of the tissue type can be received in real time by a local processor of the ultrasound scanner 101-j. At box 114, the tissue-specific preset selected based on the type of tissue identified is automatically applied to adjust the imaging parameter settings of the ultrasound scanner 101-j for subsequent live imaging (at box 116). Thus, subsequent live imaging (e.g., acquisition and display of image data) by the ultrasound scanner 101-j occurs using the automatically reconfigured imaging settings (at box 116). As described, process 100 can run in real time in the background (i.e., while a live scan of a subject is being performed) so that, as the operator moves the probe to image different anatomical structures of the subject, the system identifies the anatomical structures represented in the image data in real time, selects an appropriate TSP, and automatically applies the settings defined by the TSP to adjust the imaging parameters of the system without requiring user intervention (i.e., manual selection of a TSP or manual adjustment of individual settings), thereby freeing the user's attention to patient care.
[0031] Also like Figure 1As shown, the reconfiguration of the system based on organ detection may alternatively or additionally involve automatic reconfiguration of the system based on predicted settings. The predicted settings may be system-determined settings, for example using a properly trained neural network that corresponds to preferred or optimal settings for a particular user and / or patient. In some examples, the predicted settings may be applied after the selection of a TSP, as described above, such as to further fine-tune the imaging settings of the ultrasound scanner 101-j. In some examples, the steps of selecting and applying a TSP may be omitted, and tissue identification may be used only to initiate the settings prediction process as described herein. For example, tissue identification may be a preliminary step in the prediction process, because the type of tissue may be an input to the settings prediction model. In some examples, tissue identification may be followed by the selection of an appropriate TSP, and the settings defined by the TSP may be used as an input to the settings prediction model in lieu of or in addition to the type of tissue. The settings prediction model 120 may include at least one neural network, which implies one or more properly arranged neural networks and is trained to output imaging parameter settings corresponding to user-preferred and / or patient-specific settings. These user-specific settings, patient-specific settings, or combinations thereof (collectively referred to as predicted settings) can be automatically applied (at box 122) to the ultrasound scanner 101-j, so that subsequent live imaging of the system occurs according to the predicted settings until further adjusted by the user or the system. As described, in one example, once the tissue type is identified at box 112, the type of tissue and the user identification and patient identification information are provided to the settings prediction model (box 120). The settings prediction model 120 can be implemented using any suitable artificial neural network architecture (e.g., a machine learning regression model) that is appropriately trained to predict one or more preferred settings. In some examples, the model 120 can be trained using historical data (e.g., system logs from multiple previous scans performed by the same operator and / or one or more recurring patients). The settings prediction model 120 can be configured to receive as input any combination of one or more of the user identification information, patient identification information (as shown in box 113), tissue type, and settings defined by the TSP. The user identification information may include, for example, a user ID, which may be typed or provided by the user in some other manner (e.g., via voice input, biometric scanning, or scanning of the user's badge) such as at the beginning of the examination. The patient identification information may include any information that may be used to identify the patient, such as the patient's name, medical record ID, date of birth, etc., which may also be input by a user or otherwise received by the ultrasound system, as further described below. One or more predicted settings for one or more corresponding imaging parameters are output by the settings prediction model at block 120, and the predicted settings are automatically applied to the ultrasound scanner 101-j to affect subsequent live imaging using the ultrasound scanner 101-j.
[0032] Therefore, a method for ultrasonically examining biological tissue according to the principles of the present disclosure may include receiving, by a processor of an ultrasound system, an ultrasound image from a live ultrasound image stream in real time, receiving an identification of a type of biological tissue in the ultrasound image, selecting one of a plurality of presets stored in a memory of the ultrasound system based on the type of biological tissue in the ultrasound image, automatically adjusting one or more imaging parameters of the ultrasound system to settings defined by the selected preset based on the type of biological tissue, identifying one or more user-specific settings, patient-specific settings, or a combination thereof for at least one of one or more of the imaging parameters using an artificial neural network, and automatically adjusting at least one imaging parameter based on the one or more user-specific settings, patient-specific settings, or a combination thereof for subsequent live imaging.
[0033] Also like Figure 1 As shown, the process 100 may also include setting up training of a prediction model 120 (at block 115). The model 120 may be trained using data extracted from patient records (which may be stored in an electronic medical record database 119) and from system logs generated by a plurality of ultrasound imaging systems (e.g., ultrasound scanners 101-1, 101-2 to 101-m, possibly including ultrasound scanner 101-j). The system logs may be stored in any suitable data storage device (e.g., a log repository 117, which may be part of a medical facility's image archiving and communication system, or any combination of wired or wireless networked data storage devices).
[0034] Figure 2 A block diagram of a system 200 is shown according to some examples of the present disclosure. Figure 2 The system in may be at least partially embodied and used to perform process 100 or any sub-process thereof. Figure 2 An ultrasound data acquisition unit 210 is shown, including an ultrasound transducer or probe 211 , a beamformer 220 , a controller 224 , and a signal processor 222 . Figure 2 Also shown is a user interface 236 , which includes a display 238 , a memory 229 , and at least one image data processor 223 , all of which are communicatively coupled to the ultrasound data acquisition unit 210 , such as via a data bus 226 . Figure 2 The components of system 200 and their arrangement shown are illustrative only, and variations such as combining, rearranging, adding, or removing components are contemplated.
[0035] The ultrasound data acquisition unit 210 can be configured to acquire ultrasound image data 232, which can be displayed in real time on the display 238 in response to the processor 223 (i.e., when the image data is acquired by performing an ultrasound scan on the object). Therefore, the processor 223 can be configured to generate and cause the system (i.e., the display 238) to display a live ultrasound image stream of the biological tissue 216 in real time. The image is generated and displayed according to the imaging parameters of the ultrasound system (e.g., depth, focus, gain, dynamic range, etc.). The system can also operate in a manner (e.g., capture mode) in which a portion of the live stream is simultaneously recorded in an image playback section in the system's memory 229, for example, for future offline examinations. The system 200 can be communicatively connected (e.g., via a wired or wireless connection) to an external storage device 235 for retrieving image data or other information and for long-term storage of acquired image data (e.g., still images or image playback). In some examples, external data can be retrieved and storage can also be provided by a cloud-based computing device 239.
[0036] The ultrasound data acquisition unit 210 may include some or all of the components of a typical ultrasound scanner. For example, the ultrasound data acquisition unit 210 may include an ultrasound transducer or probe 211, which includes an ultrasound sensor array 212. The sensor array 212 is configured to transmit ultrasound 214 to biological tissue 216 (e.g., liver, kidney, breast, heart tissue, or other types of biological tissue of a subject) and detect echoes 218 therefrom to perform ultrasound imaging of the tissue 216. Different types of tissue may be scanned in different examinations, and thus the system 200 may be configured to receive (e.g., in response to user input and / or image data analysis performed by a processor) an indication of the type of tissue, and in a preferred example, the determination of the type of tissue is performed by the ultrasound system 200.
[0037] Various transducer arrays may be used, for example, linear arrays, curved arrays, or phased arrays. The array 212 may, for example, include a two-dimensional array of transducer elements that can be scanned in both elevation and azimuth dimensions for 2D and / or 3D imaging. The ultrasound data acquisition unit 210 includes a signal processor 222, which may be housed with the sensor array 212, or it may be physically separated from the sensor array 212 but communicatively coupled to the sensor array 212 (e.g., via a wired or wireless connection). For example, the array 212 may be located in a handheld probe, and the signal processor 222 may be located in an ultrasound system base 230, which in some cases may be embodied in a portable computing device such as a tablet computer.
[0038] The array 212 may be coupled to the system base 230 via a beamformer 220 configured to control the operation of the array 212. In some embodiments, the beamformer 220 may include one or more beamformers (e.g., a microwave beamformer combined with a main beamformer in the ultrasound system base, or a combination of a transmit and receive microwave beamformer and / or a main beamformer). The beamformer 220 may be configured to control the transmission of ultrasound and the reception of echo signals by the array 212. In some embodiments, the beamformer 220 may include a microwave beamformer that may be co-located with the ultrasound array in the probe and operate on a group of sensor elements for transmission and / or reception of signals by a group of sensor elements of the ultrasound sensor array 212. In some embodiments, the microwave beamformer may be coupled to a transmit / receive (T / R) switch (not shown), which may be configured to switch between transmission and reception to protect the main beamformer from high energy transmission signals. In some embodiments, such as in a portable ultrasound system, the T / R switch and other elements of the system may be included in the ultrasound probe instead of the system base 230. The ultrasound base typically includes software and hardware components, including circuits for signal processing and image data generation, and executable instructions for providing a user interface. In some embodiments, the ultrasound probe can be coupled to the ultrasound system base via a wireless connection (e.g., WiFi, Bluetooth) or via a wired connection (e.g., a probe cable that can be configured for parallel or serial data transmission).
[0039] The system 200 may include one or more processing components for generating an ultrasound image based on the echoes detected by the array 212. For example, the system 200 may include a signal processor 222 that may be configured to process the echo signals received from the transducer 211 for generating ultrasound image data, and at least one image data processor 223 for presenting the ultrasound image data (e.g., ultrasound image 232) on a display 238 of the system 200. The ultrasound data acquisition unit 210 may include or be operatively coupled to a user interface 236, which may be integrated with or otherwise physically connected to a system base 230 housing the signal processor 222. In some embodiments, at least some components of the user interface may be wirelessly connected to the signal processor 222.
[0040] The user interface 236 may include a display 238 for displaying the ultrasound image 232 and, in some cases, an interactive graphical user interface (GUI) component. The user interface 236 may also include one or more user controls 237 for controlling the operation (one or more) of the system 200. In some embodiments, the (one or more) user controls 237 may include one or more hard controls (e.g., buttons, knobs, dials, encoders, mice, trackballs, etc.) that may be provided on a control panel of the system base 230. In some embodiments, the (one or more) user controls 237 may additionally or alternatively include soft controls (e.g., GUI control elements or simply GUI controls) provided on a touch-sensitive display. The system 200 may also include a local memory 229. The local memory may be provided by one or more hard disk drives, solid-state drives, or any other type of suitable storage device including non-volatile memory. The local memory 229 may be configured to store image data, executable instructions, or any other information required for the operation of the system 200. In some examples, system 200 may also be communicatively connected (via a wired or wireless connection) to external storage (e.g., storage device 235, such as a storage device of a picture archiving and communication system (PACS), a cloud-based computing device 239, or a combination thereof).
[0041] The signal processor 222 may be communicatively, operatively and / or physically coupled to the sensor array 212 and / or the beamformer 220. The signal processor 222 may be configured to receive unfiltered and scrambled ultrasound data representing ultrasound echoes 218 detected by the sensor array 212. Based on the data, the signal processor 222 may be operable to generate ultrasound image data, which may be appropriately arranged into an image 232 for display, for example, by the processor 223. For example, the signal processor 222 may be configured to process the received echo signals in various ways, such as bandpass filtering, decimation, I and Q component separation, and harmonic signal separation. The signal processor 222 may also perform additional signal enhancement, such as speckle reduction, signal compounding, and noise elimination. The signal processor 222 may then generate B-mode image data from the component signals, for example, by employing amplitude detection or any other known or later developed technique for imaging structures in the body. The B-mode image data may be further processed by scan conversion, for example, to arrange the signals in a spatial relationship in which they are received in a desired image format. For example, scan conversion can arrange the signals into a two-dimensional (2D) sector format or a three-dimensional (3D) format of a cone or other shape. The B-mode image data can alternatively or additionally be processed by a multiplanar reformatter that is configured to convert echoes received from points in a common plane in a volumetric region of the body into an ultrasound image (e.g., a B-mode image) of that plane, such as described in U.S. Pat. No. 6,443,896 (Detmer). One or more processors (e.g., processor 222 or 223) of system 200 can additionally or alternatively generate a volume rendering of the B-mode image data (i.e., an image of a 3D data set as viewed from a given reference point), such as described in U.S. Pat. No. 6,530,885 (Entrekin et al.).
[0042] Signal processing and generation of image data may be performed in real time as the operator ultrasonically scans tissue 216 so that the image data may be displayed as a real-time (or live) image of the object. Alternatively, the image 232 may be generated based on previously acquired image data stored in a memory (e.g., local or external memory) associated with the system 200. As described above, the ultrasound data acquisition unit 210 may include a controller 224 that may be configured to set imaging parameters of the system 200, for example, to control the transmission and reception of signals by the array 212, as well as certain signal and image processing functions of the system 200. Among other things, the controller 224 may control or set imaging parameters of the system 200, which settings may be used by the beamformer 220 to control the excitation of elements of the array for transmission and detection of signals by the array 212. The settings applied by the controller 224 may also affect the signal and image processing of the acquired ultrasound data, for example, by controlling the dynamic range of compression for display of the image, or other image processing or display settings. As described, the transmission of ultrasound pulses from the transducer array 212, under the control of the beamformer, may be directed by a transmit / receive controller, which may be coupled to the T / R switch and may receive input from a user's operation of the user interface 236. In the case of an electronically steerable array, another function that may be controlled by the controller 224 is the direction in which the beam is steered. The beam may be steered straight ahead from the transducer array 212 (orthogonal to the transducer array 212), or at different angles for a wider field of view.
[0043] like Figure 2As shown, the data acquisition unit 210 can be communicatively connected to at least one processor 223, which is configured to perform one or more of the functions associated with intelligent (also known as computer-assisted or AI-assisted) scanning of biological tissues, for example, automatic and dynamic reconfiguration of imaging parameters of the system based on the scanned tissue, without the operator's involvement in the setting of the imaging parameters. The processor 223 may include one or more processing units (e.g., one or more single-core or multi-core CPUs, a single GPU or GPU cluster, or any arrangement of multiple processors configured for parallel processing, for example) and is uniquely configured to perform the functions described herein. For example, the processor 223 may also be configured to receive an ultrasound image in real time from a live ultrasound image stream acquired by the ultrasound data acquisition unit 210, receive an identification of a biological tissue type in the ultrasound image (e.g., in response to user input and / or automatic image data analysis performed by the processor 223, as further described herein), and generate at least one predicted setting of at least one of the multiple imaging parameters using an artificial neural network 228 based on the type of biological tissue. Then, the processor 223 may automatically apply the at least one predicted setting to the corresponding imaging parameter for subsequent live imaging. For example, the processor 223 can transmit commands to the controller 224 to change the acoustic settings of the system, such as depth, focus, transmit and / or receive frequency, etc., or to the signal processor 222 and / or other components of the system for converting echo signals into image data to adjust other imaging parameters (such as TGC or velocity range (e.g., in the case of Doppler imaging) to adjust the imaging parameters of the system according to the (one or more) predicted settings, so that the (one or more) predicted settings are automatically used by the system in subsequent live imaging until further adjusted by the system or the operator.
[0044] In addition to performing functions associated with intelligent scanning of biological tissue, the processor 223 can be configured to provide functions associated with the display of image data, which is relevant information. In some embodiments, the processor 223 may include a display processor 234, which may further include functions for generating and causing the display 238 to present annotations along with the image data, such as annotations identifying a selected preset, any adjustments made to the preset, such as by highlighting those imaging parameters adjusted by the artificial neural network, and / or simply listing one or more of the imaging parameters used to produce the image being displayed (e.g., in real time). The annotations may also be saved (e.g., stored with the image, or as annotations on the image and / or as metadata accompanying the stored image). In embodiments of the present invention, the display processor 234 may receive image data for further enhancement, buffering, and temporary storage before being displayed on the display 238. The display 238 may include a display device implemented using various known display technologies such as LCD, LED, OLED, or plasma display technologies. Although for purposes of illustration, the present invention is not limited to the following embodiments. Figure 2 Engine 227 and display processor 234 are shown as separate components in , but in fact the functionality of these components (and any other processing components described herein) may be integrated into a single processor or a cluster of processors arranged to operate together (eg, in parallel).
[0045] As will be further described, processor 223 may include a setup prediction engine 227, which may be implemented in any suitable combination of software (e.g., executable instructions in the form of source code or compiled / machine instructions) and hardware components (e.g., one or more processors programmable by executable instructions and / or hardwired circuits, such as an application specific integrated circuit ASIC specifically programmed to perform one or more functions of engine 227). Engine 227 may include a program for executing reference Figure 1The functions of engine 227 may be implemented via processor-executable instructions 225, which, when executed by processor 223, configure or program the processor to perform functions associated with generating predicted settings (e.g., settings determined by the system to reflect patient-specific and / or user-specific preferences) and automatically applying these settings to imaging system 200. In some embodiments, as further described herein, engine 227 may include at least one artificial network of neural perceptrons 228 (also referred to herein as an artificial neural network or simply a neural network) that is trained to perform one or more of the functions of engine 227. Any suitable type of machine learning algorithm or model (e.g., generative, discriminative, generative adversarial, classification, regression, convolutional, or any combination thereof) with any suitable architecture (e.g., configuration of interconnected layers) may be used to implement neural network 228. In some such embodiments, neural network 228 may implement a machine learning regression model that is trained based on historical data (e.g., system log files from previous scans) to identify appropriate patient-specific and / or user-specific settings for any given imaging situation. Neural network 228 can have any suitable architecture and thus include any suitable number of input layers, output layers, and hidden node layers in any suitable arrangement (e.g., convolutional layers, normalization layers, pooling layers, and / or fully connected layers). In some examples, network 228 can include an arrangement of one or more sub-networks that form a larger network that is trained to produce (one or more) desired results. In other examples, neural network 228 can be operatively associated with additional programming, such as to perform pre-processing of input data and / or post-processing of the output of network 228 for use by imaging system 200.
[0046] As described, some or all of the components of the system 200 may be co-located (e.g., within the system base 230) and communicatively connected (e.g., via the data bus 226). Additionally or alternatively, components of the system 200 may be connected to remote components via one or more wired or wireless connections. For example, the system base 230 may additionally be communicatively coupled to an external storage device 235, such as an external drive or PACS storage device of a medical institution. In some embodiments, some or all of the functionality of one or more neural networks used by the imaging system 200 may reside in a remote computing device 239, such as a cloud server, which may be communicatively coupled to the system base 230 (e.g., a portable system such as a tablet-based U / S scanner) that is configured to transmit live images to the cloud for classification and / or other analysis, and receive the output of the classification or analysis (e.g., identification of tissue type) for selection of a TSP and / or for automatic reconfiguration of the system 200.
[0047] As described herein, in some examples, identification of biological tissue can be performed by processor 223, for example using a machine learning model, such as a suitably trained machine learning organ classification model. Figure 2 Only a single neural network 228 is shown in FIG. 2 , but it will be appreciated that the processor 223 may use multiple appropriately connected neural networks, which may have different architectures and may be trained for different end results. For example, Figure 3 As shown, the processor 300 may include (or communicate with) a neural network 310 trained as an anatomical structure classifier, and another neural network 320 trained as a setting predictor (one or more). The neural network 310 may be configured to output an identification of the type of biological tissue being imaged (e.g., organ identification 315), which is coupled to the second neural network 320 for setting prediction (one or more). Tissue type identification may be performed by a suitably trained neural network (such as a convolutional neural network trained to classify ultrasound images based on the type of tissue included in the image).
[0048] In some examples, the system may optionally include a memory (e.g., local memory 229, which may include any suitable type of non-volatile memory (e.g., read-only memory, programmable read-only memory, electrically erasable programmable read-only memory, flash memory, random access memory, or any other type of suitable non-volatile memory) that stores a plurality of presets (i.e., tissue-specific presets or TSPs). Each preset may include or define an ensemble of imaging parameter settings that have been determined (e.g., through testing) to be suitable for imaging a given type of tissue. In some embodiments, the processor 223 may be configured to select one of the plurality of stored presets based on the type of biological tissue, and automatically apply the selected preset to adjust one or more of the plurality of imaging parameters of the system to the settings defined by the selected preset, e.g., before or while the predicted settings are being generated. In some examples, the settings defined by the selected preset may be used as input to the neural network 228. In such examples, the neural network 228 may be trained to output user- and / or patient-specific settings associated with any given selected TSP, and thus the neural network 228 may be viewed as customizing or fine-tuning the presets according to the preferences of the operator without any operator involvement.
[0049] Also refer to Figure 3 , an example processor 300 for an ultrasound imaging system is described. The processor 300 may be used to implement one or more of the processors (e.g., the processor 223) of an ultrasound imaging system (e.g., the system 200) according to the present disclosure. Figure 3As shown, the processor 300 receives as input an ultrasound image 304, which may be coupled to the processor 300 in real time (e.g., during live imaging). That is, the term "live ultrasound image" may be used to refer to an ultrasound image from a live image stream acquired by the system.
[0050] The processor 300 may include an anatomical structure classifier 310 configured to identify the type of biological tissue represented in the received image 304. In some examples, the anatomical structure classifier 310 is a machine learning model (e.g., such as in Figure 5 As described, the anatomical structure classifier 310 is configured to identify the type of biological tissue represented in the ultrasound image 304 and transmit the identification 315 to the setting prediction model 320. In some embodiments, such as in the case of a machine learning model, the anatomical structure classifier 310 can be trained to classify the input ultrasound image into one or more possible tissue or organ classifications. In other embodiments, tissue identification can be obtained differently, for example, by any suitable image processing technique other than machine learning (e.g., segmentation), or in response to user input (e.g., by operator voice input to auditorily identify the anatomical structure being scanned when the operator moves the probe to a new position).
[0051] Tissue type identification may be coupled to a settings prediction model 320 that is configured to output a set of imaging parameters that are predicted or determined by the processor, for example based on training, to be suitable for imaging that type of tissue. Thus, in a preferred embodiment, settings prediction model 320 may be implemented using at least one appropriately trained machine learning model(s), such as using a machine learning regression model (e.g., as described in reference Figure 6-9 The setting prediction model 320 may receive as input(s) any combination of tissue type, user identification information, patient identification information, and one or more current imaging settings of the system. Figure 3In the example of , the setting prediction model 320 receives as input a tissue type 315 as well as (one or more) user identification inputs 306, (one or more) patient identification inputs 308, and current settings (e.g., TSP settings 312 selected by the system automatically or in response to user input). As described herein, the setting prediction model 320 is trained to output one or more predicted settings 314, which can then be automatically applied to reconfigure the ultrasound imaging system, such as by coupling them to one or more controllers 301 of the system. When the processor 300 performs tissue type identification and setting prediction in the background, the need for operator involvement in setting appropriate imaging parameters can be avoided and / or the processor can be used to double-check that appropriate imaging settings have been selected. In the latter case, if the processor 300 determines that a different set of imaging parameter settings should be applied instead of the selected settings, and the processor 300 is not operating in the automatic reconfiguration mode, the system (e.g., system 200) can additionally optionally provide user feedback to warn the user that they may be imaging with incorrect settings.
[0052] In some embodiments, the patient identification input(s) 306 may be obtained according to any of the examples in the co-pending patent application entitled “ULTRASOUND SYSTEM WITH ARTIFICIAL NEURAL NETWORK FOR RETRIEVING IMAGING PARAMETER SETTINGS FOR RECENT PATIENTS”, the contents of which are incorporated herein by reference in their entirety for any purpose. For example, the processor 300 may additionally implement or be communicatively coupled to at least one additional neural network that is trained to identify recurring patients based on patient identification information received by the system. Recurring patients may be associated with a patient ID and thus the patient identification input 308 may correspond to the patient ID. The settings prediction model 320 may therefore be operable to output a predicted setting 314 based at least in part on the patient identification input 308. In some examples, in response to user input, the patient ID and / or other patient identification information (e.g., height, weight, BMI, name, date of birth, age, gender, medical record ID, etc.) may be provided to the system (e.g., to the processor 300). In some examples, the setting prediction model 320 can be configured to receive any combination of a patient ID and / or other patient identification information (e.g., height, weight, BMI, name, date of birth, age, gender, medical record ID, etc.) as the patient identification input(s) 308, and output a predicted setting based at least in part on the patient identification input(s) 308. In some embodiments, the setting prediction model 320 can include a cascade network (i.e., a plurality of appropriately connected sub-networks), portions of which are trained to perform patient identification based on patient identification information (e.g., patient height, patient weight, patient BMI, patient name, date of birth, age, gender, patient medical record ID, etc.), and which are operatively connected to other portions of the cascade network to supply the patient identification thereto for setting prediction.
[0053] Return to reference Figure 3 , the processor 300 may be configured to optionally select a pre-stored TSP (at selection box 316) based on the tissue type identified by the anatomical structure classifier 310. In some examples, the processor 300 may be configured to retrieve the selected TSP from a memory 318 (e.g., a local memory of the ultrasound scanner) and cause the settings defined by the selected TSP (TSP settings 312) to be applied to the system, such as by coupling corresponding commands to the controller(s) 301. In some examples, the processor 300 may be configured to alternatively or additionally couple the TSP settings 312 as input to a settings prediction model 320. Thus, in some examples, the settings prediction model 320 may be configured to output predicted settings 314 also based in part on the current settings of the system (in which case it would be the settings defined by the previously selected TSP). In some such examples, the setting prediction model 320 can be configured to receive an input vector having dimension n+i, where n corresponds to the number of settings defined by the TSP and i corresponds to the number of additional input variables such as one or more user identification parameters (e.g., alphanumeric user ID, biometric ID input, etc.) and one or more patient identification parameters (e.g., patient name, medical record ID, patient gender, age, weight, etc.), and generate an output vector having dimension n.
[0054] Figure 4 A block diagram of a process for training and deploying a neural network in accordance with the principles of the present disclosure is shown. Figure 4 On the left hand side of , Stage 1 illustrates the training of the neural network. Training may involve selection of a starting network architecture 412 and preparation of training data 414. The starting network architecture 412 may be a blank architecture (e.g., an architecture with a defined arrangement of layers and nodes but without any previously trained weights) or a partially trained network, such as an initial network, which may then be further customized for classification of ultrasound images. The starting architecture 412 (e.g., blank weights) and the training data 414 are provided to a training engine 410 to train the model. After a sufficient number of iterations (e.g., when the model performs consistently within an acceptable error), the model 420 is said to be trained and ready for deployment, as shown in FIG. Figure 4 As shown in the middle of the figure, stage 2. Figure 4On the right-hand side or stage 3, the trained model 420 is applied (via an inference engine 430) to analysis of new data 432, which is data that has not been presented to the model during the initial training (stage 1). For example, the new data 432 may include unknown images, such as live ultrasound images acquired during a scan of a patient. The trained model 420 implemented via the engine 430 is used to classify the unknown images based on the training of the model 420 to output predictions 434 (e.g., classification(s) of the type of tissue represented in the input image). The system can then use the predictions 434 (e.g., the type of biological tissue) for subsequent processes 440 (e.g., as input to one or more other machine learning models and for implementing actions by the system, such as automatic reconfiguration of the imaging parameters of the system).
[0055] In examples where the training model 420 is a classification model, the starting architecture can be that of a convolutional neural network or a deep convolutional neural network that can be trained to perform image classification, image segmentation, image comparison, or any combination thereof. With increasing amounts of stored medical image data (e.g., in PACS or cloud storage), the availability of high-quality clinical images is increasing, which can be used to train a neural network to learn the probability that a given image contains a given tissue type. The training data 414 can include multiple (hundreds, often thousands, or even more) annotated / labeled images, also referred to as training images. It will be understood that the training images need not include a complete image produced by the imaging system (e.g., representing the complete field of view of the probe), but can include patches or portions of the image that label biological tissue types.
[0056] Figure 5 An example architecture of a neural network 500 that can be trained to perform ultrasound image classification according to the principles of the present disclosure is shown. The neural network 500 may include any suitable number and arrangement of layers, including an input layer 510, a plurality of intermediate or hidden layers 512 (e.g., convolutional, normalization, pooling, fully connected or other types of layers), and an output layer 514. The input layer 510 is configured to receive an input corresponding to an ultrasound image 502. For example, in the case of a convolutional network, the intermediate layers 512 are configured to apply a plurality of filters to the input (one or more) of each successive layer starting from the layer to generate a stack of feature maps, which can then be provided to a fully connected output layer 512, which is configured to output the classification of the image as one of a plurality of possible categories 518 or the probability that the image falls into each of the possible categories. Thus, in some examples, the output layer 514 provides an output vector 504 having a dimension equal to the number of possible categories 518 (e.g., liver, kidney, heart, lung, breast, fetus, blood vessel, etc.). In some examples, the network 500 may be configured to provide, for example, only a single classification corresponding to the classification with the highest probability as an output.
[0057] Any suitable architecture, such as ResNet, AlexNet, VGGNet, GoogLeNet, or others, can be used as a starting architecture for neural network 500. In some examples, the starting architecture (e.g., Figure 4 412) may be a "blank slate" network that can be trained from scratch (i.e., without any pre-configured weights). Figure 4 As described, the starting architecture and training dataset can be provided to a training engine to train a network for a specific task, in this case, for classifying ultrasound images into one of a plurality of possible tissue type categories or classifications. In some examples, the starting architecture can be a partially trained network, such as an Inception v3 network trained on the ImageNet dataset, which can be used as a starting point and then supplementally trained or fine-tuned with an additional training dataset that includes only ultrasound images, thereby reducing the overall training time to obtain an appropriately trained model. Thus, supplementary training of a previously trained network architecture (e.g., Inception v3) involves refining the weights of the network and dense (fully connected) layers to provide a multi-label classifier configured to generate an output vector of length equal to the number of different tissue types represented in the training dataset.
[0058] In some instances, depending on the particular architecture of the neural network 500, the ultrasound image retrieved from the live stream can require preprocessing before it can be coupled to the neural network (e.g., 500) for classification. For example, when using a network of the Inceptionv3 architecture, the live ultrasound image can first be preprocessed (e.g., reproduced in triplicate) to provide a 3-channel input as required by the Inception v3 architecture. In some examples, the triple representation of the incoming image can include three different representations of the entire dynamic range of the original acquired image (e.g., high signal with a low dynamic range image, low signal with a low dynamic range image, and a compressed high dynamic range image with an adaptive histogram equal to the entire dynamic range or otherwise defined). In some examples, the ultrasound system can natively output multi-channel (color or grayscale) images, in which case the preprocessing step can be omitted, or a different preprocessing step can be performed. Therefore, depending on the architecture deployed in the field (e.g., on the ultrasound system 200), the processor communicating with the neural network may or may not require a preprocessing block, as described herein. As further described, the neural network 500 may include or be operatively connected to a post-processing block, such as for selecting one of a plurality of tissue type categories (e.g., the category associated with the highest probability value), which may then be provided as input to a downstream processing block (e.g., Figure 3 320 and / or selector block 316). Although Figure 5An example network architecture is shown in , but it will be understood that in other examples, other suitable architectures may be used to implement the tissue type detection component of the process.
[0059] Figure 6 An example architecture for a neural network 600 is shown, which may be used to implement the settings prediction model 320. The neural network 600 may be configured to perform regression to arrive at one or more predicted settings 622. For example, the neural network may include a plurality of fully connected layers (610-1 to 610-n), wherein a first fully connected layer serves as an input layer configured to receive an input vector 612. The last (or output) layer of the network 600 is a regression layer 620, which is configured to generate an output vector 622, the individual elements of which define predicted preferred settings for individual imaging parameters of the system.
[0060] 7-9 illustrate various aspects of training a machine learning regression model (e.g., neural network 600) according to the principles of the present disclosure. As described, the model can be trained using settings extracted from system log files. For example, Fig. 7A A portion of a system log 700 is shown, such as a utilization log generated by an ultrasound imaging system sold under the trade name EPIQ by PHILIPS Ultrasound. The system log can track any adjustments made to the system, such as changing imaging settings. This portion of the log 700 shows recorded information 710, 712 associated with a given user action, such as recording the type of user action (e.g., preset_activate 710 to indicate selection of a preset) and the specific preset selected (e.g., adult echo 712). Figure 7B A portion of another system log 720 is shown, such as a workflow log generated by an EPIQ system. The portion of log 720 shows changes in depth parameters, such as depth change events 722-1 and 722-2, where corresponding depth values 724-1 and 724-2 are set to 1 and 2, respectively.
[0061] To prepare training data, you can build a time series graph based on data extracted from system logs. For example, refer to Figure 8, curves 802 may be generated for individual imaging parameters of the system (e.g., depth, focus, gain, transmit / receive frequency, velocity range, dynamic range, etc.). Each of the curves 802 plots the change in the value of the corresponding parameter as a function of time. In the illustrated example, curve 802-1 shows a time series log of depth settings, curve 802-2 shows the change in focus settings over time, and curve 802-3 shows the change in gain settings over time. The time series logs may be examined manually or with the assistance of a computer to identify preferred settings for a particular combination of user and patient. For example, the curves may be examined to identify a plateau in the curves, which may indicate that the user has reached a preferred setting. Additionally or alternatively, the curves may be examined just prior to an image freeze, record, and / or archive operation to identify the value of a given imaging parameter, which may therefore represent a preferred setting for a particular combination of user and patient. This data extraction process (which may be manual or at least partially computer automated) may be performed for hundreds, thousands, or more combinations of users and patients to compile a training data set. In some examples, the training data may be tabulated, such as Fig. 9 Thus, each row 904 (e.g., observations X1 to X i ) can correspond to a unique combination of input parameters of a setting prediction model that can be used to train the model. As noted, table 902 can contain hundreds, thousands, or even more rows 904 of training data. In some examples, a portion of the tabulated data (e.g., up to 5%, up to 10% or more) can be retained as validation data and / or test data for use in validating and / or testing the performance of the model before deployment. As described, patient information data can alternatively or additionally be extracted from other sources, such as from an electronic medical record (EMR) system and / or from annotations or metadata archived with stored ultrasound images associated with the system log. Training data, such as input parameter combinations of individual rows, can be provided to the model, and the weights of the model can be iteratively updated (e.g., by backpropagation), and after training is complete, the trained model can be tested to confirm the performance of the model at or above the expected baseline accuracy level, at which point the model is deemed to have been properly trained (e.g., training model 420) and ready for deployment (e.g., in an inference or prediction engine 430 such as the setting prediction engine 227 of the system 200).
[0062] Examples herein can improve previously known ultrasound imaging systems in that they provide processor-implemented techniques for automatically reconfiguring an imaging system to acquire better quality images. According to examples herein, a system can automatically and dynamically reconfigure itself (e.g., apply better-suited imaging parameters) in response to an operator scanning different regions of an object. This automatic reconfiguration is achieved by one or more background processes that analyze image data to identify the type of tissue being imaged, automatically select and apply an appropriate TSP for that tissue type, and also automatically adjust or adapt settings defined by the selected TSP to settings determined by the system as preferred for the specific user and / or patient being imaged.
[0063] Although examples of generating medical images from sparsely sampled data are described herein with reference to ultrasound image data, it will be appreciated that the examples herein are equally applicable to training a neural network to generate images from a sparse data set of any imaging modality, such as magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET), and virtually any other imaging modality.
[0064] In various embodiments where components, systems and / or methods are implemented using programmable devices such as computer-based systems or programmable logic, it should be appreciated that the above-described systems and methods may be implemented using various known or later developed programming languages such as "C", "C++", "FORTRAN", "Pascal", "VHDL", etc. Accordingly, various storage media may be prepared, such as magnetic computer disks, optical disks, electronic memory, etc., which may contain information that may direct a device such as a computer to implement the above-described systems and / or methods. Once the appropriate device accesses the information and programs contained on the storage medium, the storage medium may provide the information and programs to the device, thereby enabling the device to perform the functions of the systems and / or methods described herein. For example, if a computer disk containing appropriate material (such as source files, object files, executable files, etc.) is provided to a computer, the computer may receive the information, configure itself appropriately and perform the functions of the various systems and methods outlined in the above figures and flow charts to implement the various functions. That is, the computer may receive various portions of information relating to different elements of the above-described systems and / or methods from the disk, implement the individual systems and / or methods and coordinate the functions of the individual systems and / or methods described above.
[0065] In view of the present disclosure, it should be noted that the various methods and devices described herein can be implemented in hardware, software and firmware. In addition, various methods and parameters are included only by example rather than in any limiting sense. In view of the present disclosure, those of ordinary skill in the art can implement this teaching to determine their own technology and the equipment required to implement these technologies while remaining within the scope of the present invention. The functions of one or more of the processors described herein can be incorporated into a smaller number or a single processing unit (e.g., CPU), and can be implemented using a dedicated integrated circuit (ASIC) or a general-purpose processing circuit that is programmed to perform the functions described herein in response to executable instructions.
[0066] Although examples of the present system may have been described with particular reference to ultrasound imaging systems, it is also contemplated that the present system may be extended to other medical imaging systems in which one or more images are obtained in a systematic manner. Thus, the present system may be used to obtain and / or record image information related to, but not limited to, the kidneys, testicles, breasts, ovaries, uterus, thyroid, liver, lungs, musculoskeletal, spleen, heart, arteries, and vascular systems, as well as other imaging applications related to ultrasound-guided interventions. In addition, the present system may also include one or more programs that can be used with conventional imaging systems so that they can provide the features and advantages of the present system. Certain additional advantages and features of the present disclosure will be apparent to those skilled in the art after studying the present disclosure, or may be experienced by a person employing the novel systems and methods of the present disclosure. Another advantage of the present system and method may be that conventional medical imaging systems can be easily upgraded to incorporate the features and advantages of the present system, apparatus, and method.
[0067] Of course, it will be appreciated that any of the examples, embodiments, or processes described herein may be combined with one or more other examples, embodiments, and / or processes, or separated and / or performed among separate devices or device portions in accordance with the present systems, devices, and methods.
[0068] Finally, the above discussion is intended only to illustrate the present system and should not be construed as limiting the claims to any particular embodiment or group of embodiments. Therefore, although the present system has been described in particular detail with reference to exemplary embodiments, it should be appreciated that many modifications and alternative embodiments may be devised by those of ordinary skill in the art without departing from the broader and intended spirit and scope of the present system as set forth in the following claims. Accordingly, the specification and drawings are to be viewed in an illustrative manner and are not intended to limit the scope of the claims.
Claims
1. An ultrasound imaging system with automatic setting of imaging parameters during live imaging, the system include: a probe configured to transmit ultrasound toward a recurrence patient to generate an ultrasound image of a biological tissue of the recurrence patient; a processor configured to generate, according to a plurality of imaging parameters of the ultrasound imaging system, and to enable the ultrasound imaging system to display a live ultrasound image stream of the biological tissue in real time, wherein the processor is further configured to: receiving an ultrasound image in real time from the live ultrasound image stream; receiving an identification of a type of the biological tissue in the ultrasound image; receiving patient identification information, user identification information, or a combination thereof; generating at least one predicted setting for at least one imaging parameter of the plurality of imaging parameters for the recurrent patient based on the type of the biological tissue and the patient identification information, the user identification information, or a combination thereof; and The at least one predicted setting is applied to corresponding imaging parameters for subsequent live imaging of the recurring patient.
2. The system according to claim 1, in, The processor is configured to generate the at least one predicted setting using an artificial neural network model, wherein the artificial neural network model is a machine learning regression model, and wherein the artificial neural network model is a model trained using historical data extracted from system logs from multiple previous scans performed by the same user and / or the same patient.
3. The system according to claim 2, in, The regression model was trained using data extracted from system logs from multiple ultrasound imaging systems.
4. The system according to claim 1, in, The processor is configured to generate the at least one predicted setting further based on a current setting for the at least one imaging parameter of the plurality of imaging parameters.
5. The system according to claim 4, in, The processor is configured to couple a set of input parameters to the deep learning model, the set of input parameters comprising: tissue type identification, multiple current settings for imaging parameters of the system, and at least one user identification input and at least one patient identification input, the multiple current settings comprising a current depth setting, a current gain setting, and a current focus setting.
6. The system according to claim 1, in, To generate the at least one predicted setting, the processor is configured to execute a trained neural network model, the trained neural network model comprising a plurality of layers, the plurality of layers comprising a first input layer and an output layer, the first input layer being configured to receive an input of size n+i, the output layer being configured to generate an output of size n, and wherein n is equal to the number of settings defined by the selected preset.
7. The system according to claim 6, in, The trained neural network model includes multiple intermediate fully connected layers, and wherein the output layer is a fully connected regression layer.
8. The system according to claim 1, in, The processor is further configured to automatically determine the type of the biological tissue in the ultrasound image upon receiving the ultrasound image from the live ultrasound image stream.
9. The system according to claim 8, in, The processor is configured to use a machine learning classification model to identify the type of the biological tissue.
10. The system according to claim 9, in, The processor is configured to identify the type of the biological tissue using a machine learning classification model having a convolutional neural network architecture.
11. The system of claim 2, further comprising a memory storing a plurality of presets, each preset defining one or more settings for at least one of the plurality of imaging parameters of the ultrasound imaging system, and in, The processor is further configured to: selecting one of a plurality of stored presets based on the type of the biological tissue; and The one or more settings of the selected preset are provided as input to the artificial neural network model.
12. The system according to claim 11, in, The processor is further configured to automatically adjust corresponding one or more imaging parameters of the plurality of imaging parameters of the ultrasound imaging system to settings defined by the selected preset before or while generating the at least one predicted setting.
13. A method for real-time ultrasonic imaging of biological tissue, the method include: receiving, by a processor of the ultrasound system, an ultrasound image in real time from a live ultrasound image stream for a recurring patient, wherein an ultrasound image including the ultrasound image in the live ultrasound image stream is generated according to current settings of a plurality of imaging parameters of the ultrasound system; receiving an identification of a type of the biological tissue in the ultrasound image; receiving patient identification information, user identification information, or a combination thereof; generating a predicted setting for at least one of the plurality of imaging parameters for the recurrent patient based on the type of the biological tissue and the patient identification information, the user identification information, or a combination thereof; and The current setting of the at least one imaging parameter of the plurality of imaging parameters is automatically adjusted based on the predicted setting for subsequent live imaging of the recurring patient.
14. The method according to claim 13, in, The generating includes coupling the type of the biological tissue and the patient identification information, the user identification information, or a combination thereof to an artificial neural network, the artificial neural network including an input layer, a plurality of intermediate layers, and an output regression layer, the input layer being configured to receive a multi-dimensional input vector, the output regression layer being configured to output a multi-dimensional output vector having a smaller dimension than the input vector, wherein the artificial neural network is a model trained using historical data extracted from a system log from multiple previous scans performed by the same user and / or of the same patient.
15. The method according to claim 14, in, The output layer and the intermediate layers are fully connected layers.
16. The method according to claim 14, further comprising: include: Selecting one of a plurality of presets from a memory of the ultrasound system, each preset defining one or more settings for at least one of the plurality of imaging parameters of the ultrasound system; and providing the one or more settings of the selected preset as input to the artificial neural network.
17. The method of claim 16, further comprising automatically adjusting one or more of the plurality of imaging parameters of the ultrasound system to settings defined by the selected preset before or while generating the at least one predicted setting.
18. The method of claim 13, further comprising determining, by the processor in response to receiving the ultrasound image from the live ultrasound image stream, the type of the biological tissue in the ultrasound image.
19. The method according to claim 18, in, The determining includes coupling the ultrasound image from the live ultrasound image stream to another artificial neural network, the other artificial neural network being trained to perform organ classification on the input ultrasound image.
20. A non-transitory computer readable medium comprising executable instructions which, when executed, cause a processor of a medical imaging system to perform any of the methods of claims 13-19.
21. A method of developing and deploying a system for live ultrasound imaging according to any of the methods of claims 13-19, the method further comprising training a neural network model to generate the predicted settings and storing the trained neural network model as processor executable instructions in a computer readable medium of the system, in, The training includes: extracting a plurality of user preference settings from a system log of at least one ultrasound imaging system; generating a plurality of sets of training data, each set comprising at least one user preferred setting from the plurality of user preferred settings and a user identification parameter associated with the at least one user preferred setting; performing training of the neural network model using a first subset of the training data; and A second subset of the training data is used to validate performance of the trained neural network model.
