Automatic network selection for ultrasound

Abstract

Ultrasound imaging devices, systems, and methods are provided. A system is configured to process a newly acquired object image by implementing a neural network specifically trained to process the object image. Image acquisition settings, such as gain and / or scan depth, for imaging an object are used to determine which wider object category corresponds to the object. Determining an appropriate category triggers the implementation of a neural network specifically associated with that category, facilitating the input of images acquired from the object into the neural network for processing. A plurality of neural networks may be used to perform a given function using the acquired ultrasound images, and each neural network may be trained using only a subset of images corresponding to a unique object category. Each network can thus be smaller, faster, more efficient than a large network required to process images from a wider range of objects.

Description

Technical Field

[0001] The present disclosure relates to ultrasound systems and methods for automatic neural network selection based on classification of an object to be imaged. Implementations include determining an object class corresponding to the object being imaged and inputting images obtained therefrom into a class-specific neural network for processing. Background Art

[0002] Artificial intelligence algorithms are being developed to support ultrasound systems in automated procedure guidance, workflow automation, and image interpretation assistance. In particular, convolutional neural networks can be relied upon to perform complex tasks in essentially real time using ultrasound images acquired during the execution of scanning protocols. Due to the diversity of images that the network is expected to process correctly at the application stage, a wide variety of images are typically required to train such networks. This diversity stems primarily from user-dependent manual image acquisition, as well as the variety of anatomical and physiological characteristics found in different objects, such as medical patients. Object diversity further increases the variability of acquired images, which requires increasingly larger training data sets and complex neural networks. Developing and training sufficiently robust networks to provide accurate results for a variety of possible input images is both expensive and challenging, not to mention the speed required to execute in real time on existing ultrasound system hardware with limited storage space. These obstacles are particularly evident for handheld ultrasound systems running on mobile devices, which are increasingly common in clinical care applications.

[0003] Improved techniques are therefore needed to rapidly process large numbers of diverse ultrasound images using neural networks. Summary of the invention

[0004] The present disclosure describes an ultrasound system configured to classify an imaged object based at least in part on ultrasound system settings used to acquire images therefrom. The disclosed system can input newly acquired images into class-specific neural networks, each neural network being trained using a set of different images corresponding only to its assigned class. Each class can be defined by a unique set of criteria such that images assigned to a particular class are more similar to each other than images assigned to different classes. Each neural network can be selected from a plurality of neural networks, wherein each neural network is trained to process images of different classes. By classifying similar images into different classes and assigning only one class to one neural network, each neural network can be trained using a smaller set of images relative to the number of images required to train a neural network responsible for processing a larger number of images with a greater variety of features. Using smaller, more compact networks, each dedicated to one class of objects, can increase processing speed and reduce computational load, requiring little user intervention, thereby achieving automated, real-time image processing and reducing data storage.

[0005] According to at least one example disclosed herein, an ultrasound imaging system may include: an ultrasound transducer configured to acquire echo signals in response to an ultrasound pulse emitted toward an object; and one or more processors in communication with the ultrasound transducer. The processor may be configured to determine an object class corresponding to the object based on an inspection parameter, the inspection parameter including one or more image acquisition settings for generating at least one image frame based on the acquired echo signals. The processor may also be configured to select a neural network associated with the object class from a plurality of neural networks, each neural network being associated with a different object class, and input at least one image frame to the neural network, the neural network being configured to generate an output.

[0006] In some examples, the system also includes a user interface configured to receive an indication of the inspection parameters. In some examples, the system also includes a controller coupled to the ultrasonic transducer, the controller being configured to implement one or more image acquisition settings in the ultrasonic transducer. In some examples, the inspection parameters also include one or more characteristics of the object. In some examples, the one or more characteristics of the object include one or more of body mass index (BMI), weight, developmental stage, age, height, or health status. In some examples, the inspection parameters also include an inspection type, which includes one or more of the following: lung ultrasound, cardiac ultrasound, abdominal ultrasound, obstetric ultrasound, pelvic ultrasound, transabdominal ultrasound, transvaginal ultrasound, vascular ultrasound, liver ultrasound, kidney ultrasound, transcranial ultrasound, or thyroid ultrasound. In some examples, the one or more image acquisition settings include gain, scanning depth, probe type, frequency, or scanning mode.

[0007] In some examples, the user interface also includes a display configured to display outputs. In some examples, the outputs include: one or more ultrasound images, identification of natural or artificial features visible within the ultrasound images, or user instructions for performing an ultrasound examination. In some examples, the one or more processors are configured to determine the object category by applying one or more thresholds to the examination parameters. In some examples, the one or more thresholds include a range of depth settings included in the image acquisition settings. In some examples, each of the multiple neural networks is operatively associated with a training algorithm, the training algorithm being configured to receive a series of training inputs and known outputs, wherein the training inputs include ultrasound image frames obtained from different object categories.

[0008] According to at least one example disclosed herein, a method of ultrasound imaging involves acquiring echo signals in response to ultrasound pulses emitted by a transducer operatively coupled to an ultrasound system into a target region of an object. The method also includes generating at least one image frame based on the echo signals and determining an object class based on examination parameters, the examination parameters including one or more image acquisition settings for generating at least one image frame based on the acquired echo signals. The method also includes selecting a neural network associated with the object class from a plurality of neural networks associated with different object classes, inputting at least one image frame into a neural network configured to generate an output, and causing the output to be displayed on a user interface.

[0009] In some examples of the method, the inspection parameters also include one or more characteristics of the object. In some examples of the method, the one or more characteristics of the object include one or more of body mass index (BMI), weight, developmental stage, age, height, or health status. In some examples of the method, the image acquisition settings include gain, scanning depth, probe type, frequency, or scanning mode. In some examples of the method, determining the object category includes applying one or more thresholds to the inspection parameters. In some examples of the method, the one or more thresholds include a range of depth settings included in the image acquisition settings. In some examples, the method also includes training each of a plurality of neural networks using an array of training inputs and known outputs, the training inputs including ultrasound image frames obtained from different object categories.

[0010] Any method or steps thereof described herein may be embodied in a non-transitory computer-readable medium containing executable instructions which, when executed, may cause one or more hardware processors to perform the method or steps embodied herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram illustrating an example of an ultrasound system configured for automatic neural network selection according to an embodiment of the present disclosure.

[0012] Figure 2 is a block diagram of an example processor illustrating an example implementation according to an embodiment of the present disclosure.

[0013] Figure 3 is a block diagram illustrating additional components of an ultrasound system configured for automatic neural network selection in accordance with an embodiment of the present disclosure.

[0014] Figure 4 is a block diagram illustrating an example of an automatic neural network selection process and associated ultrasound system components for implementing the process according to an embodiment of the present disclosure.

[0015] Figure 5 is a flow chart of an automatic neural network selection method implemented according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0016] Various embodiments are described in more detail below with reference to the accompanying drawings, which form a part of the embodiments and show specific exemplary embodiments. However, the embodiments can be implemented in a variety of different forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided to make the present disclosure thorough and complete and to fully convey the scope of the present disclosure to those skilled in the art. Each embodiment can be practiced as a method, system, or device. Therefore, the embodiments can take the form of a hardware implementation, a complete software implementation, or an implementation combining software and hardware aspects. Therefore, the following detailed description should not be interpreted as restrictive.

[0017] References in the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present invention. The appearance of the phrase "in one embodiment" at various locations in the specification does not necessarily refer to the same embodiment.

[0018] As used herein, the term "plurality" may refer to two or more items, components, or features. For example, a plurality of neural networks may include two, three, four, five, six or more neural networks.

[0019] Some of the following descriptions are presented with respect to symbolic representations of operations on non-transient signals stored in computer memories. These descriptions and representations are means used by those skilled in the art of data processing to most effectively convey the essence of their work to other persons skilled in the art. Such operations generally require physical manipulations of physical quantities. These quantities may take the form of electrical, magnetic, or optical signals, and may be stored, transmitted, combined, compared, or otherwise manipulated. Sometimes, primarily for reasons of common use, these signals are referred to as bits, values, elements, symbols, characters, terms, numbers, and the like. In addition, sometimes, for convenience, specific arrangements of steps requiring physical manipulations of physical quantities are also referred to as modules or code devices, without loss of generality.

[0020] However, all of these and similar terms will be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise specifically stated as apparent from the following discussion, it should be understood that discussions throughout this specification using terms such as "processing," "computing," "calculating," "determining," "displaying," etc. refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical electronic quantities within a computer system memory or registers or other such information storage, transmission, or display device.

[0021] As used herein, a "subject" may refer to a patient receiving outpatient or inpatient medical care, which may involve a medical procedure or examination, so that in some examples, a "class of subjects" may include a class of patients. It is important to note, however, that the term "subject" is not limited to medical patients; it may also refer to a human or animal being imaged by an imaging system for a variety of reasons and in a variety of settings.

[0022] The ultrasound system according to the present disclosure may implement one or more neural networks, such as at least one convolutional neural network (CNN), deep neural network (DNN), etc., which may be uniquely synchronized with the data acquisition hardware and user interface. Various embodiments may utilize a set of two or more neural networks trained to perform the same or similar functions, but using different categories of ultrasound images as input. Systems that are not configured as disclosed herein may need to create and implement neural networks that are configured to generate the same output according to a specific examination, but use one or more additional processing steps that are necessary due to differences between images acquired by different subject groups and different ultrasound operators. Since the implementation (one or more) of the disclosed system can incorporate or be compatible with various trained neural networks that perform various functions, the functions performed by a given set of neural networks may vary. For example, a neural network may be trained to recognize specific features present in an ultrasound image, such as anatomical features, ultrasound artifacts (e.g., A lines or B lines), medical devices, etc. In addition or alternatively, a neural network may be trained to recognize certain features present in an acquired ultrasound image, and in response, generate instructions for adjusting the ultrasound probe position and / or image acquisition settings to obtain images with higher clarity, images taken along a specific scanning plane, or images required for a specific scanning protocol.

[0023] Neural networks used in accordance with the present disclosure may be hardware (e.g., neurons represented by physical components) or software-based (e.g., neurons and paths implemented in a software application), and may use a variety of topologies and learning algorithms for training to produce desired outputs. For example, a software-based neural network may be implemented using a processor configured to execute instructions (e.g., a single-core or multi-core CPU, a single GPU or GPU cluster, or multiple processors arranged for parallel processing), the instructions may be stored in a computer-readable medium, and the instructions, when run, cause the processor to perform various functions and generate corresponding outputs for additional processing and display. The ultrasound system may include a display or graphics processor operable to arrange ultrasound images and / or additional graphical information (which may include a work list of features to be imaged and / or measured, annotations, tissue information, patient information, indicators, and other graphical components) in a display window for display on a user interface of the ultrasound system. In some embodiments, the ultrasound images and associated information may be provided to a storage device and / or memory device, such as a picture archiving and communication system (PACS), for reporting purposes, progress tracking, or future machine training (e.g., to continue to improve the performance of each neural network). In some examples, ultrasound images obtained during scanning can be selectively or automatically transmitted via a communications network to a specialist trained to interpret the information embodied in the images, such as an obstetrician-gynecologist, ultrasound specialist, physician, or other clinician, thereby allowing users to perform ultrasound scans in various locations.

[0024] An ultrasound system according to the principles of the present disclosure may include or be operatively coupled to an ultrasound transducer configured to transmit ultrasound pulses toward a medium, such as a human body or a particular portion thereof, and to generate echo signals in response to the ultrasound pulses. The ultrasound system may include a beamformer configured to perform transmit and / or receive beamforming, and a display configured to display ultrasound images and associated outputs generated by the ultrasound imaging system.

[0025] Certain aspects of the invention include process steps and instructions that can be implemented in software, firmware, or hardware, and when implemented in software, can be downloaded to reside on and be operated by different platforms used by various operating systems. Embodiments may include one or more applications available via the Internet, such as software as a service (SaaS), which can be accessed using various computer devices (e.g., smartphones, tablets, desktop computers, etc.).

[0026] Figure 1An example ultrasound system 100 implemented in accordance with the principles of the present disclosure is shown. The ultrasound system 100 may include an ultrasound data acquisition unit 110 including an ultrasound probe including an ultrasound sensor array 112 configured to transmit ultrasound pulses 114 to a region of an object 116 (e.g., a chest region of a person) and to receive ultrasound echoes 118 in response to the transmitted pulses. As further shown, the ultrasound data acquisition unit 110 may include a beamformer 120 and a signal processor 122, which may be configured to generate a stream of discrete ultrasound image frames 124 from the ultrasound echoes 118 received at the array 112. The system 100 may also include a data processor 126, such as a computing module or circuit, configured to determine an object class based on ultrasound system settings applied to acquire image frames 124 from the object 116 and optionally one or more features of the object. Based on the determined object class, the data processor 126 (alone or in combination with one or more additional processors) may select a newly acquired image frame 124 and input it into one of a plurality of neural networks 128a, 128b, 128c that is specifically trained to process images from that particular object class. The neural networks 128a, 128b, 128c may be configured to use the input image frame 124 to perform a variety of functions, non-limiting examples of which may include identifying natural or man-made features visible within the image frame, generating user instructions for acquiring additional image frames, etc. Since each neural network 128a, 128b, 128c may be trained to specifically process image frames from one object class, the network may be more compact and faster than a large network trained to process a wider selection of image frames from a variety of different object classes. Automatically determining the object class and selecting the appropriate neural network accordingly may eliminate or at least reduce the need for user intervention in the image processing process, which may require extensive validation studies and regulatory documentation.

[0027] As described above, all neural networks 128a, 128b, 128c can be trained to perform the same function, such that the image frame 124 is processed in the same or similar manner regardless of the network. Embodiments of the system disclosed herein can be configured to implement various different groups of neural networks, for example, a first group of neural networks 128a, 128b, 128c is configured to perform one function, such as identifying a certain anatomical feature within an acquired image frame, a second group of neural networks 130a, 130b, 130c is configured to perform a different function, such as identifying a certain (one or more) ultrasound artifacts from the acquired image frame, a third group of neural networks 132a, 132b, 132c is configured to perform yet another different function, such as generating an alarm based on the identified anatomical feature or artifact, and so on. In some embodiments, automatic selection and implementation of a group of similarly trained neural networks from a larger group of neural networks can be initiated upon receipt of an indicated user input. For example, selecting a lung imaging protocol may cause newly acquired image frames to be input into a set of neural networks trained to identify the pleural line, while selecting a fetal imaging protocol may cause newly acquired image frames to be input into another set of neural networks trained to identify various anatomical features of a developing fetus. Other non-limiting examples of exam types may include: cardiac ultrasound, abdominal ultrasound, obstetric ultrasound, pelvic ultrasound, transabdominal ultrasound, transvaginal ultrasound, transrectal ultrasound, vascular ultrasound, liver ultrasound, kidney ultrasound, transcranial ultrasound, and / or thyroid ultrasound.

[0028] In operation, the data processor 126 may determine the object class based on the ultrasound imaging settings used to image the particular object 116. These settings may include depth, gain, focal area, and / or frequency, among others. The relevant settings may also include probe types, such as linear, sector, or curved, which differ in overall image geometry and beamforming, resulting in differences in contrast and resolution of the resulting image at depth. The object class may also be determined using object characteristics or demographic data, non-limiting examples of which may include: body mass index (BMI), weight, age, height, developmental stage, and / or one or more health conditions, among other factors. One or more object characteristics may be input by a user or may be collected from an electronic medical record, for example, if the ultrasound system being used is connected to a hospital IT system. In general, the factors used to image the object may correspond to a particular object class associated with a neural network that is pre-trained using image frames obtained from a sample of images obtained from a number of objects belonging to the object class. Thus, the selected neural network is particularly suitable for processing a certain subset of images.

[0029] In some embodiments, object class determination may be performed by thresholding individual ultrasound settings or combinations of settings. For example, for lung ultrasound examinations, three neural networks may be trained: one for infant subjects, one for pediatric subjects, and one for adult subjects. Mainly due to the average differences in physical size between these objects, the depth settings used to image them may be different. Different depth settings may be used to organize the imaged objects by category, which may then be used to input images acquired therefrom into a neural network assigned to a particular category. Classifying and pooling images in this manner may be more efficient, faster, and more accurate than using a single neural network to process all images from two or more categories.

[0030] The actual thresholds used to classify objects may vary depending on the type of examination. By way of example only, a depth setting of less than 5 centimeters for a lung ultrasound examination may assign subsequently acquired images to an infant class, a depth setting between 5 centimeters and 10 centimeters may assign images to a pediatric object class, and a depth setting greater than 10 centimeters may assign images to an adult object class. The same or similar threshold limits may be used to acquire additional image frames during the same examination so that newly acquired images are assigned to the neural network in a consistent manner. Advantageously, the system may be configured to "dynamically" redirect newly acquired images obtained from the same object into different neural networks as ultrasound imaging settings are modified. This real-time or substantially real-time adjustment is achieved at least in part by utilizing multiple smaller, more efficient neural networks trained for a particular set of images.

[0031] Similarly, body type or BMI can be used to classify objects into different categories. Images collected from high BMI objects are typically noisier and of relatively lower quality due to the attenuation of the subcutaneous fat layer. This attenuation typically changes the appearance of the underlying anatomical structures of the object, so that separate neural networks may be required to process images obtained from low BMI, normal, and high BMI objects. Increased gain settings can be applied to compensate for the higher attenuation typical of high BMI objects. Gain settings can then be used to distinguish object categories defined at least in part by body type, and can be further refined by additional ultrasound settings and / or object characteristics.

[0032] If it is determined that the final output or overall performance of one or more neural networks is not optimal, the front-end thresholds used to determine object class may be modified so that the image frames received by the underperforming (one or more) neural networks may change. For example, by examining the accuracy and quality of ultrasound images and associated (one or more) outputs obtained from objects of known classes, the thresholds used to determine object class may be adjusted over time to improve the final system output. Figure 1The one or more data processors and / or neural networks depicted in can be configured to adjust various threshold limits in response to the final output evaluation, thereby creating an automatic feedback loop that iteratively modifies the threshold limits in a manner that enhances the accuracy, sensitivity, and / or quality of the final output. Similar to the object class determination and neural network selection, the iterative refinement of the threshold limits can be performed automatically by one or more system components with little or no user intervention. In some examples, image statistics such as average brightness and signal-to-noise ratio can be used in conjunction with gain settings to determine and refine the identified body type, for example, because body types with higher BMIs tend to correspond to darker and noisier images.

[0033] Various ultrasound systems (e.g., Philips LUMIFY) can be utilized and / or configured to implement the disclosed methods in real time or substantially real time. Mobile, handheld, or other point-of-care ultrasound systems can be used to perform the disclosed methods and can exhibit significant performance improvements due to the use of smaller, more specifically trained neural networks (each neural network requires less storage space relative to existing neural networks used for similar ultrasound image processing). In some examples, image processing and resultant output can be implemented on a system separate from the components dedicated to ultrasound image acquisition, such as a cloud-based system to which ultrasound images and ultrasound settings are transmitted wirelessly or via a wired connection. In some embodiments, one or more acquisition components (e.g., ultrasound data acquisition unit 110) can be uniquely coupled to the components used for image processing.

[0034] Figure 2 1 is a simplified block diagram illustrating an example processor 200 in accordance with the principles of the present disclosure. One or more processors for implementing the disclosed methods (such as the processor 126 of the system 100) may be configured the same as or similar to the processor 200. The processor 200 may be used to implement one or more processes described herein, such as automatically determining object categories (before and after starting an ultrasound examination), automatically selecting and / or implementing a neural network, and / or automatically adjusting object category attribute thresholds.

[0035] Processor 200 can be any suitable processor type, including but not limited to a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable array (FPGA) where the FPGA has been programmed to form a processor, a graphics processing unit (GPU), an application specific circuit (ASIC) where the ASIC is designed to form a processor, or a combination thereof.

[0036] Processor 200 may include one or more cores 202. Core 202 may include one or more arithmetic logic units (ALUs) 204. In some examples, core 202 may include a floating point logic unit (FPLU) 206 and / or a digital signal processing unit (DPU) 208 in addition to or in lieu of ALU 204.

[0037] Processor 200 may include one or more registers 212 communicatively coupled to core 202. Registers 212 may be implemented using dedicated logic gates (e.g., flip-flops) and / or any memory technology. In some examples, registers 212 may be implemented using static memory. Registers may provide data, instructions, and addresses to core 202.

[0038] In some examples, processor 200 may include one or more levels of cache memory 210 communicatively coupled to core 202. Cache memory 210 may provide computer readable instructions to core 202 for execution. Cache memory 210 may provide data for processing by core 202. In some examples, computer readable instructions may have been provided to cache memory 210 by a local memory (e.g., a local memory attached to external bus 216). Cache memory 210 may be implemented with any suitable cache memory type, such as metal oxide semiconductor (MOS) memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), and / or any other suitable memory technology.

[0039] The processor 200 may include a controller 214 that may control inputs to one or more processors included herein (e.g., the processor 126). The controller 214 may control data paths in the ALU 204, FPLU 206, and / or DSPU 208. The controller 214 may be implemented as one or more state machines, data paths, and / or dedicated control logic. The gates of the controller 214 may be implemented as independent gates, FPGAs, ASICs, or any other suitable technology.

[0040] Registers 212 and cache 210 may communicate with controller 214 and core 202 via internal connections 220A, 220B, 220C, and 220D. The internal connections may be implemented as buses, multiplexers, crossbar switches, and / or any other suitable connection technology.

[0041] Input and output of processor 200 may be provided via bus 216, which may include one or more conductors. Bus 216 may be communicatively coupled to one or more components of processor 200, such as controller 214, cache 210, and / or registers 212. Bus 216 may be coupled to one or more components of the system.

[0042] The bus 216 may be coupled to one or more external memories. The external memory may include a read-only memory (ROM) 232. The ROM 232 may be a mask ROM, an electrically programmable read-only memory (EPROM), or any other suitable technology. The external memory may include a random access memory (RAM) 1333. The RAM 233 may be a static RAM, a battery-backed static RAM, a dynamic RAM (DRAM), or any other suitable technology. The external memory may include an electrically erasable programmable read-only memory (EEPROM) 235. The external memory may include a flash memory 234. The external memory may include a magnetic storage device, such as a disk 236.

[0043] Figure 3 is a block diagram of another ultrasound system 300 in accordance with principles of the present disclosure. Figure 3 One or more of the components shown may be included in a system configured to determine the object category based on the image acquisition settings and then implement an appropriate neural network to process the newly acquired image. For example, one or more of the above functions of the data processor 126 and / or the ultrasound data acquisition unit 110 may be performed by Figure 3 One or more processing components are shown implemented and / or controlled, including, for example, a signal processor 326 , a B-mode processor 328 , a scan converter 330 , a multi-planar reformatter 332 , a volume renderer 334 , and / or an image processor 336 .

[0044] exist Figure 3In the system, the ultrasound probe 312 includes a transducer array 314 for transmitting ultrasound to a target area of ​​an object according to a scanning protocol and receiving echo information in response to the transmitted waves. In various embodiments, the transducer array 314 can be a matrix array or a one-dimensional linear array. The transducer array 314 can be coupled to a microbeamformer 316 in the probe 312, and the microbeamformer 1116 can control the transmission and reception of signals by the transducer elements in the array. In the example shown, the microbeamformer 316 is coupled to a transmit / receive (T / R) switch 318 via a probe cable, which switches between transmission and reception and protects the main beamformer 322 from high-energy transmission signals. In some embodiments, the T / R switch 318 and other elements in the system can be included in the transducer probe rather than in a separate ultrasound system component. The transmission of the ultrasound beam from the transducer array 314, which can be under the control of the microbeamformer 316, is directed by a transmit controller 320 coupled to the T / R switch 318 and the beamformer 322, which receives input from, for example, a user's operation of a user interface or control panel 324. A function that can be controlled by the transmit controller 320 is the direction of beam steering. The beam can be steered to be vertically forward (perpendicular to the transducer array) from the transducer array, or at a different angle for a wider field of view. The partially beamformed signal produced by the microbeamformer 316 is coupled to the beamformer 322, where the partially beamformed signals from the individual patches of transducer elements are combined into a fully beamformed signal.

[0045] The beamformed signal may be transmitted to a signal processor 326. The signal processor 326 may process the received echo signals in various ways, such as bandpass filtering, decimation, I and Q component separation, and / or harmonic signal separation. The processor 326 may also perform signal enhancement by ripple reduction, signal compounding, and / or noise elimination. In some examples, the data generated by the different processing techniques employed by the signal processor 326 may be used by a data processor and / or at least one neural network to identify one or more anatomical features, ultrasound artifacts, and / or image views. The processed signal may be coupled to a B-mode processor 328, which may employ amplitude detection to image structures in the body. The signal generated by the B-mode processor 328 may be coupled to a scan converter 330 and a multi-plane reformatter 332. The scan converter 330 may arrange the echo signals in a desired image format according to the spatial relationship in which the echo signals are received. For example, the scan converter 330 may arrange the echo signals into a two-dimensional (2D) sector-shaped format. The multi-planar reformatter 332 can convert echoes received from points in a common plane in a volumetric region of the body into an ultrasound image of that plane, as described in U.S. Pat. No. 6,443,896 (Detmer). In some examples, the volume renderer 334 can convert echo signals of a 3D data set into a projected 3D image as seen from a given reference point, for example, as described in U.S. Pat. No. 6,530,885 (Entrekin et al.). The 2D or 3D image can be transmitted from the scan converter 330, the multi-planar reformatter 332, and the volume renderer 334 to the image processor 336 for further enhancement, caching, and / or temporary storage for display on the image display 337.

[0046] Prior to display, each image may be input and processed by one of a plurality of neural networks 338 that is trained to process each image from a particular object class and generate one or more outputs based on each image. In embodiments, each of the plurality of neural networks 338 may be implemented at various processing stages, for example, prior to processing performed by the image processor 336, the volume renderer 334, the multi-planar reformatter 332, and / or the scan converter 330.

[0047] The graphics processor 340 can generate graphic overlays for display with the processed ultrasound images. These graphic overlays can include, for example, standard identification information, such as the patient's name, the date and time of the image, imaging parameters, etc., as well as various outputs generated by each neural network 338, such as one or more indicators that convey the presence, absence, and / or marking of one or more anatomical features embodied in the current image, and / or whether various anatomical features have been observed and / or measured, and / or which anatomical features have not yet been observed and / or measured according to a stored work list. The graphic overlays can also include visual instructions, such as text and / or symbols, for guiding the user of the system 300 to perform the ultrasound scan in a manner required to obtain the images and / or measurements required for a particular evaluation. In some examples, the graphics processor can receive input from the user interface 324, such as a typed patient name or confirmation that the user of the system 300 has confirmed and / or implemented the instructions displayed or issued from the interface. The user interface 324 may also receive input regarding characteristics of the subject, selection of a particular imaging mode and the operating parameters contained in such a modality, input prompting adjustments to settings and / or parameters used by the system 300, input requesting additional instructions or assistance in performing an ultrasound scan, and / or input requesting that one or more ultrasound images be saved and / or transmitted to a remote receiver. The user interface 324 may also be coupled to a multiplanar reformatter 332 for selecting and controlling the display of a plurality of multiplanar reformatted (MPR) images.

[0048] Figure 4 An example method 400 of ultrasound imaging, object class determination, and automatic network selection, and ultrasound system components for performing the method are depicted. As shown, the method 400 may involve initiating an ultrasound examination at step 402, and selecting ultrasound imaging settings at step 404. One or both of steps 402, 404 may be performed via a user interface 406 coupled to an ultrasound image acquisition unit. A patient or object class (class A, B, or C) may then be determined in step 408 based on the selected ultrasound settings, which may be used in combination with one or more object characteristics. A pre-trained neural network (network A, B, or C) uniquely associated with the patient or object class may then be determined in step 410, as well as one or more parameters for processing image frames acquired according to the method 400. As further shown, at least one data processor 412 may be used to perform steps 408 and / or 410.

[0049] Image acquisition begins at step 414, and the next image is acquired using ultrasound image acquisition unit 418 at step 416. Each acquired image is then assigned to a neural network corresponding to an object class and processed according to the selected scanning protocol at step 420. Processing each acquired image may involve several sub-processes. For example, at step 422, each image may be pre-processed using parameters specific to the object class and associated neural network. Non-limiting, non-exhaustive pre-processing steps may involve normalizing scaling values ​​and / or adjusting image input size for neural network compatibility. The pre-processed image is then processed using the specified neural network at step 424 and post-processed using an additional set of parameters associated with the object class at step 426. In some embodiments, post-processing may involve converting, formatting, and / or adjusting the network processed images and / or probabilities according to user and / or system specifications. One or more of the processing steps included in step 420 may be performed by another data processor 428 or the same processor 412 used to determine the object class and associated neural network. If more images are needed or desired, the method returns to step 416 and repeats from there. If additional image acquisition is not required, the results may be displayed in step 430 on a user interface 432, which may include the same user interface 406 used to enter initial ultrasound settings. The neural network output may be displayed after processing each individual image frame, or after accumulating results for a sufficient number of images for display, e.g., all frames from a short movie loop clip. In some embodiments, the method 400 may also involve refining and ultimately optimizing the thresholds used for object class determination, which may improve classification and processing of newly acquired images.

[0050] Figure 5 is a flow chart of an ultrasound imaging method performed in accordance with the principles of the present disclosure. Example method 500 illustrates steps that may be utilized by the systems and / or devices described herein, in any order, for determining an object class and triggering implementation of a particular neural network trained to process acquired image frames corresponding to the object class. Method 500 may be performed by an ultrasound imaging system (e.g., system 100) or other systems, including, for example, a mobile system (e.g., LUMIFY by Royal Philips plc ("Philips")). Additional example systems may include SPARQ and / or EPIQ, also produced by Philips.

[0051] In the illustrated embodiment, the method begins at block 502, "Acquiring echo signals in response to ultrasound pulses transmitted to a target region of a subject by a transducer operatively coupled to an ultrasound system."

[0052] In block 504, the method involves "generating at least one image frame based on the echo signal."

[0053] At block 506, the method involves "determining an object class based on examination parameters including one or more image acquisition settings for generating image frames from acquired echo signals."

[0054] At block 508, the method involves "selecting a neural network associated with an object class from a plurality of neural networks associated with different object classes."

[0055] At block 510, the method involves "inputting an image frame to a neural network, the neural network configured to generate an output."

[0056] At block 512, the method involves "causing output to be displayed on a user interface."

[0057] Of course, it should be understood that, according to the present systems, devices and methods, any of the examples, embodiments or processes described herein may be combined with one or more other examples, embodiments and / or processes, or be separated and / or performed in discrete devices or device portions.

[0058] In various embodiments where programmable devices such as computer-based systems or programmable logic are used to implement the components, systems and / or methods, it should be understood that the above systems and methods can be implemented using any of a variety of known or later developed programming languages, such as "C", "C++", "FORTRAN", "Pascal", "VHDL", etc. Thus, various storage media, such as magnetic computer disks, optical disks, electronic memory, etc., can be prepared, which can contain information that can instruct devices such as computers to implement the above systems and / or methods. Once the information and programs contained on the storage medium are accessible to the appropriate device, the storage medium can provide the information and programs to the device, thereby enabling the device to perform the functions of the systems and / or methods described herein. For example, if a computer disk containing appropriate materials (such as source files, object files, executable files, etc.) is provided to a computer, the computer can receive the information, configure itself appropriately and perform the functions of the various systems and methods depicted in the above diagrams and flow charts to implement various functions. That is, the computer can receive various portions of information related to different elements of the above systems and / or methods from the disk, implement individual systems and / or methods, and coordinate the functions of the individual systems and / or methods described above.

[0059] Although the disclosed system may have been described with particular reference to an ultrasound imaging system, it is also contemplated that the system may be extended to other medical imaging systems that obtain one or more images in a systematic manner. Thus, the system may be used to acquire and process images and associated information related to, but not limited to, the testicles, breasts, ovaries, uterus, thyroid, liver, lungs, musculoskeletal, spleen, heart, arterial and vascular systems, and other imaging applications associated with ultrasound-guided interventions. In addition, the system may also include one or more programs that can be used with conventional imaging systems so that they can provide the features and advantages of the system. Upon studying the disclosure, certain other advantages and features of the disclosure may be apparent to those skilled in the art, or may be experienced by those who employ the novel systems and methods of the disclosure. Another advantage of the system and method may be that a conventional medical imaging system may be easily upgraded to incorporate the features and advantages of the system, apparatus, and method.

[0060] Ultimately, the above discussion is intended to be merely illustrative of the system of the present invention and should not be construed as limiting the appended claims to any particular embodiment or group of embodiments. Thus, although the present system has been described in detail with reference to exemplary embodiments, it should also be appreciated that numerous variations and alternative embodiments may be devised by those skilled in the art without departing from the broader spirit and scope of the present system as set forth in the claims. Accordingly, the specification and drawings should be regarded in an illustrative manner and are not intended to limit the scope of the appended claims.

Claims

1. An ultrasonic imaging system, comprising: an ultrasonic transducer configured to acquire an echo signal in response to an ultrasonic pulse transmitted toward the object; as well as one or more processors in communication with the ultrasound transducer and configured to: determining an object class corresponding to the object based on examination parameters, the examination parameters comprising one or more image acquisition settings for generating at least one image frame from the acquired echo signals; selecting a neural network associated with the object class from a plurality of neural networks associated with different object classes; and The at least one image frame is input to the neural network, which is configured to generate an output. 2 . The system of claim 1 , further comprising a user interface configured to receive an indication of the inspection parameter.

3. The system of claim 1, further comprising a controller coupled to the ultrasound transducer, the controller configured to implement the one or more image acquisition settings in the ultrasound transducer.

4. The system according to claim 1, wherein: The inspection parameters also include one or more characteristics of the object.

5. The system according to claim 4, wherein: The one or more characteristics of the subject include one or more of: body mass index (BMI), weight, developmental stage, age, height, or health condition.

6. The system according to claim 1, wherein: The examination parameters also include an examination type, which includes one or more of the following: lung ultrasound, cardiac ultrasound, abdominal ultrasound, obstetric ultrasound, pelvic ultrasound, transabdominal ultrasound, transvaginal ultrasound, vascular ultrasound, liver ultrasound, kidney ultrasound, transcranial ultrasound, or thyroid ultrasound.

7. The system according to claim 1, wherein: The one or more image acquisition settings include gain, scan depth, probe type, frequency, or scan mode.

8. The system according to claim 2, wherein: The user interface also includes a display configured to display the output.

9. The system according to claim 1, wherein: The output includes one or more of: an ultrasound image, an identification of natural or artificial features visible in the ultrasound image, or user instructions for performing an ultrasound examination.

10. The system according to claim 1, wherein: The one or more processors are configured to determine the object class by applying one or more thresholds to the inspection parameters.

11. The system according to claim 10, wherein: The one or more thresholds include a range of depth settings included in the image acquisition settings.

12. The method according to claim 1, wherein: Each of the plurality of neural networks is operatively associated with a training algorithm configured to receive an array of training inputs and known outputs, wherein the training inputs include ultrasound image frames obtained from different object categories.

13. A method of ultrasonic imaging, the method comprising: acquiring echo signals in response to ultrasound pulses transmitted into a target region of a subject by a transducer operatively coupled to the ultrasound system; generating at least one image frame according to the echo signal; determining the object class based on examination parameters including one or more image acquisition settings for generating the at least one image frame from the acquired echo signals; selecting a neural network associated with the object class from a plurality of neural networks associated with different object classes; inputting the at least one image frame to the neural network, the neural network configured to generate an output; and The output is caused to be displayed on a user interface.

14. The method according to claim 13, wherein: The inspection parameters also include one or more characteristics of the object.

15. The method according to claim 14, wherein: The one or more characteristics of the subject include one or more of: body mass index (BMI), weight, developmental stage, age, height, or health condition.

16. The method according to claim 13, wherein: The image acquisition settings include: gain, scanning depth, probe type, frequency, or scanning mode.

17. The method according to claim 13, wherein: Determining the object class includes applying one or more thresholds to the inspection parameters.

18. The method according to claim 17, wherein: The one or more thresholds include a range of depth settings included in the image acquisition settings.

19. The method of claim 13, further comprising training each of the plurality of neural networks using an array of training inputs and known outputs, the training inputs comprising ultrasound image frames obtained from different object categories.

20. A non-transitory computer readable medium comprising executable instructions which, when executed, cause one or more processors to perform the method according to any one of claims 13-19.

Images