System and method for automated ultrasound image labeling and quality grading

Abstract

An automated ultrasound image labeling and quality grading system and method is provided. The ultrasound system includes an ultrasound imaging device configured to acquire ultrasound images of a patient. An anatomy recognition and labeling module receives the acquired ultrasound images from the ultrasound imaging device and automatically recognizes anatomical structures within the received ultrasound images. The anatomy recognition and labeling module automatically labels the anatomical structures within the images with information identifying the anatomical structures. The acquired ultrasound images and the labeled anatomical structures are displayed on a display of the ultrasound imaging device.

Description

[Technical Field]

[0001] The present disclosure generally relates to ultrasound imaging systems and methods, and more specifically to artificial intelligence-based networks for ultrasound imaging and ultrasound image evaluation, as well as systems and methods for automatically recognizing and labeling anatomical structures in acquired ultrasound images and grading the image quality of acquired ultrasound images. [Background Art]

[0002] Ultrasound imaging is typically performed in a clinical setting by a trained ultrasound specialist. For diagnostic ultrasound imaging, specific views of an organ or other tissue or body features (such as body fluid, bone, joints, etc.) are clinically significant. Such views may be defined in clinical standards as the views that a sonographer should capture depending on the target organ, diagnostic purpose, and the like.

[0003] The image quality of an acquired ultrasound image varies depending on various factors including, for example, probe positioning, imaging parameters (e.g., depth, gain, etc.), and the like. For clinical use (e.g., for diagnosis), an ultrasound image should generally have adequate image quality. Clinicians generally require considerable training to assess the diagnostic quality of ultrasound images. Such images may be obtained in real-time during image acquisition, or may have been acquired previously. In both cases, clinicians need to understand the level of diagnostic quality of the ultrasound image. Similarly, even in training and educational settings, expert ultrasound users are required to grade the diagnostic quality of images acquired by students and novice users, which is very time-consuming for ultrasound experts.

[0004] Furthermore, considerable training is generally required for clinicians to be able to recognize anatomical structures present in ultrasound images. This is particularly difficult during real-time ultrasound image acquisition, where the ultrasound image changes continuously in real-time as the position and orientation of the probe relative to the target organ changes.

[0005] While conventional ultrasound imaging systems may be suitable for most patients in hospitals or similar clinical settings, such systems require considerable training to operate and to properly capture clinically desirable views. Furthermore, conventional ultrasound imaging devices cannot be properly operated without adequately trained professionals, increasing the overall cost of such ultrasound imaging and further limiting patients' access to ultrasound imaging. [Overview of the project]

[0006] This disclosure provides systems and methods for facilitating automated ultrasound image labeling and automated ultrasound image quality grading. In particular, the systems and methods provided herein are operable to recognize anatomical structures in acquired ultrasound images and label the recognized anatomical structures with information identifying the anatomical structures. The labels may be displayed on a display device together with the acquired ultrasound images, for example, by overlaying the labels on the ultrasound image at a location or region corresponding to the location or region of the recognized anatomical structure. Furthermore, the systems and methods provided herein are operable to automatically grade the image quality of acquired ultrasound images, the grades may be displayed or otherwise provided to the user, and in some embodiments, the grades may be used to help guide the user to acquire higher quality images, such as ultrasound images representing a clinically desirable view of an organ or other bodily feature.

[0007] In various embodiments, machine learning techniques are used to automatically grade the diagnostic quality of ultrasound images, thereby solving the problems that (i) new and novice ultrasound users are unaware of the diagnostic quality of the images being acquired, and (ii) expert instructors must spend considerable time grading the diagnostic quality of images acquired by new / novice users. Embodiments provided herein apply advanced machine learning approaches to automatically grade the diagnostic quality of ultrasound images, and the grading can be based on well-established image quality scales or standards provided by the clinical community.

[0008] In various embodiments, the problem of correctly identifying anatomical structures in ultrasound images is solved by applying machine learning algorithms to automatically recognize and label such anatomical structures in real time, during or after acquisition, or both. Advanced machine learning approaches are applied in various embodiments not only to recognize major anatomical structures in the image but also to localize them, i.e., to determine the location of each anatomical structure in the image.

[0009] In at least one embodiment, an ultrasound system is provided that includes an ultrasound imaging device and an anatomical structure recognition and labeling circuit. The ultrasound imaging device acquires an ultrasound image of a patient. The anatomical structure recognition and labeling circuit receives the acquired ultrasound image, automatically recognizes one or more anatomical structures in the received ultrasound image, and automatically labels one or more anatomical structures in the image with information identifying one or more anatomical structures. The ultrasound imaging device includes a display that shows the acquired ultrasound image and the one or more labeled anatomical structures.

[0010] In at least one embodiment, a method is provided comprising: receiving an ultrasound image acquired by an ultrasound imaging device by an anatomical structure recognition and labeling circuit; automatically recognizing one or more anatomical structures in the received ultrasound image by the anatomical structure recognition and labeling circuit; automatically labeling one or more anatomical structures in the acquired ultrasound image with information identifying one or more anatomical structures by the anatomical structure recognition and labeling circuit; and displaying the acquired ultrasound image and the one or more labeled anatomical structures.

[0011] In at least one embodiment, the ultrasound system includes an ultrasound image grading circuit configured to receive acquired ultrasound images from an ultrasound imaging device and to automatically grade the quality of the received ultrasound images. It also includes a display configured to simultaneously display the acquired ultrasound image and an indication of the quality grade. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing an automated ultrasound image labeling and quality grading system according to one or more embodiments of the present disclosure. [Figure 2] This block diagram shows the training of a machine learning circuit of the system shown in Figure 1, according to one or more embodiments of the present disclosure. [Figure 3] This is a block diagram showing a neural network that can be implemented by a machine learning circuit according to one or more embodiments of the present disclosure. [Figure 4] This is a schematic diagram of an ultrasonic imaging device according to one or more embodiments of the present disclosure. [Figure 5] This figure shows automatically labeled ultrasound images according to one or more embodiments of the present disclosure. [Figure 6A] This figure shows an ultrasound image, including a grade indicating the quality of the ultrasound image, according to one or more embodiments. [Figure 6B]This figure shows an ultrasound image, including a grade indicating the quality of the ultrasound image, according to one or more embodiments. [Modes for carrying out the invention]

[0013] This disclosure provides several embodiments of systems and methods for automated ultrasound image labeling and quality grading, as well as systems and methods for ultrasound image recognition. The systems and methods provided herein may be particularly useful for ultrasound imaging performed by novice ultrasound technicians and / or for ultrasound imaging utilizing handheld or mobile ultrasound imaging devices that may be deployed in locations other than conventional clinical settings. Utilizing an artificial intelligence approach, the systems and methods provided herein are capable of automatically recognizing and labeling anatomical structures within acquired ultrasound images. The labels can be displayed together with the ultrasound image, for example, superimposed on the corresponding anatomical structures in the image. The artificial intelligence approach is also utilized in the systems and methods provided herein for automatically determining the quality grade of acquired ultrasound images, and in some embodiments, the determined quality grade can be used to guide the user to acquire specific ultrasound images, such as a particular clinically desirable or standard view.

[0014] In various embodiments, the systems and methods provided herein may be further used to determine whether the acquired ultrasound images accurately depict or represent a desired view of a patient's organ or other tissue, a target feature or area of ​​the patient.

[0015] The systems and methods provided herein may provide feedback to the user, for example, to indicate the determined image quality of the acquired ultrasound image and whether a desired view of the patient's organs or other tissues or features has been captured. In some embodiments, the ultrasound image is displayed along with labels applied to the recognized anatomical structures within the ultrasound image.

[0016] Figure 1 shows a block diagram of an automated ultrasound image labeling and quality grading system 100 (which may be referred to herein as the ultrasound system 100) according to an embodiment of the present disclosure. As shown in Figure 1, the ultrasound system 100 includes an ultrasound imaging device 110, a communication network 102, a machine learning circuit 105, and an image knowledge database 122. Each of these may be incorporated into a single ultrasound device, such as a handheld or portable device, or may constitute a plurality of devices that are operably linked or linkable to one another. As will be described in more detail herein, the machine learning circuit 105 may include an ultrasound image recognition module 120, an anatomical structure recognition and labeling module 130, and an ultrasound image grading module 140, each of which may include a program and / or hardwired circuit configured to perform the function or action of the respective module, as described herein.

[0017] The ultrasound imaging device 110 is any ultrasound device capable of operating to acquire ultrasound images of a patient, and in at least some embodiments, it may be, for example, a handheld ultrasound imaging device. The ultrasound imaging device 110 may include a display 112, memory 114, and one or more processors 116. The ultrasound imaging device 110 is operably coupled to an ultrasound probe 118.

[0018] Memory 114 is or may include any computer-readable storage medium, such as read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive, optical storage device, magnetic storage device, electrically erasable programmable read-only memory (EEPROM), or organic storage medium.

[0019] The processor 116 may be any computer processor capable of operating to execute instructions (for example, stored in memory 114) for performing the functions of the ultrasound imaging device 110, as described herein.

[0020] The ultrasound probe 118 is driven by the ultrasound imaging device 110 to transmit a signal toward a target area of ​​the patient and to receive echo signals returning from the target area in response to the transmitted signal. In operation, the user of the ultrasound device 110 can hold the probe 118 relative to the patient's body at a position and angle to acquire a desired ultrasound image. The signal received by the probe (i.e., the echo signal) is communicated to the ultrasound imaging device 110 and may form or be processed to form an ultrasound image of the patient's target area. Furthermore, the ultrasound image may be provided to a display 112 that can display the ultrasound image and / or any other relevant information to the user.

[0021] The ultrasound images thus acquired by the ultrasound imaging device 110 can be provided to the machine learning circuit 105 via the communication network 102. The ultrasound images from the ultrasound imaging device 110 are provided to the machine learning circuit 105 as indicated by reference number 101. The communication network 102 can utilize one or more protocols to communicate over one or more physical networks, including local area networks, wireless networks, dedicated lines, intranets, and the Internet.

[0022] In one or more embodiments, the machine learning circuit 105 (including, for example, an ultrasound image recognition module 120, an anatomical structure recognition and labeling module 130, and an ultrasound image grading module 140) may be provided within the ultrasound imaging device 110, or a local copy of the machine learning circuit 105 and / or ultrasound image knowledge stored in the image knowledge database 122 may be included in the ultrasound imaging device 110, and the ultrasound imaging device 110 may access the machine learning circuit 105 located at a remote location (e.g., stored in one or more server computers, or in a "cloud").

[0023] The machine learning circuit 105 may be or include any electrical circuit configured to perform the ultrasound image recognition, image labeling, and image grading techniques described herein. In some embodiments, the machine learning circuit 105 includes, or may be executed by, a computer processor, microprocessor, microcontroller, or the like configured to perform the various functions and operations described herein with respect to the machine learning circuit 105. For example, the machine learning circuit 105 may be executed by a computer processor selectively activated or reconfigured by a stored computer program, or may be a specially constructed computing platform for implementing the features and operations described herein. In some embodiments, the machine learning circuit 105 is configured to execute software instructions stored on any computer-readable storage medium including, for example, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drives, optical storage devices, magnetic storage devices, electrically erasable programmable read-only memory (EEPROM), organic storage media, and the like.

[0024] The machine learning circuit 105 receives ultrasound images acquired from an ultrasound imaging device 110, automatically determines an image quality grade for each of the received ultrasound images, and automatically labels one or more anatomical structures in the received ultrasound images. For example, in some embodiments, an anatomical structure recognition and labeling module 130, which may be included as part of the machine learning circuit 105, automatically recognizes anatomical structures in an ultrasound image and automatically associates a label with the recognized anatomical structure. In some embodiments, the label associated with the recognized anatomical structure is displayed superimposed on or embedded in the ultrasound image (e.g., on the display 112) within a region where the corresponding anatomical structure is presented.

[0025] In some embodiments, an ultrasound image grading module 140, which may be included as part of the machine learning circuit 105, automatically determines an image quality grade for each of the received ultrasound images.

[0026] In some embodiments, an ultrasound image recognition module 120, which may be included as part of the machine learning circuit 105, automatically determines whether one or more of the received ultrasound images represent a clinically desired view of a patient's organ or other aspect, region or feature.

[0027] Each of the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140 is implemented by a computational intelligence system employing artificial intelligence drawn from an image knowledge database 122, enabling them to perform the functions of these modules described herein (for example, determining whether a received ultrasound image represents a clinically desirable view, recognizing and labeling anatomical structures in an ultrasound image, and determining the image quality grade of an ultrasound image). Some or all of the functions of the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140 described herein may be performed automatically by the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140, for example, in response to the reception of an acquired ultrasound image.

[0028] In this specification, “artificial intelligence” is used broadly to describe any computational intelligence system and method that can learn knowledge (for example, based on training data) and use such learned knowledge to adapt an approach to solve one or more problems. An artificial intelligence machine can solve problems such as image recognition, anatomical structure recognition and labeling, and image quality grading by employing, for example, neural networks, deep learning, convolutional neural networks, Bayesian programming learning techniques. Furthermore, artificial intelligence may include one or a combination of computational techniques such as constraint programming, fuzzy theory, classification, conventional artificial intelligence, symbolic processing, fuzzy set theory, evolutionary computation, cybernetics, data mining, approximate inference, derivative-free optimization, decision trees, and / or soft computing. By employing one or more computational intelligence techniques, the machine learning circuit 105 (including, for example, an ultrasound image recognition module 120, an anatomical structure recognition and labeling module 130, and an ultrasound image grading module 140) can learn to adapt to unknown and / or changing environments for better performance.

[0029] The image knowledge database 122 may contain various information that facilitates image analysis of received ultrasound images, provided by the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140.

[0030] In some embodiments, the image knowledge database 122 may include information about various image views of various organs. For example, the image knowledge database 122 may include information associated with clinically standard or preferred views of the heart. Clinically standard views of the heart may include, for example, parasternal, subcostal, short-axis and long-axis parasternal, 2-ventricular apex, 3-ventricular apex, 4-ventricular apex, and 5-ventricular apex views. Furthermore, the information associated with clinically standard views may include information associated with three-dimensional views, two-dimensional cross-sectional views, and / or sets of two-dimensional cross-sectional views.

[0031] The image knowledge database 122 may be stored in any computer-readable storage medium accessible by the machine learning circuit 105, which may include, for example, an ultrasound image recognition module 120, an anatomical structure recognition and labeling module 130, and an ultrasound image grading module 140.

[0032] Figure 2 is a block diagram illustrating the training of the machine learning circuit 105 according to one or more embodiments. In various embodiments, the training of the machine learning circuit 105 may include separate or simultaneous training of each of the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140. Furthermore, in some embodiments, each of the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140 may be implemented as a separate machine learning model, while in other embodiments, some or all of the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140 may be implemented with the same machine learning model.

[0033] The machine learning circuit 105 can be trained based on training images 210. Training images 210 may include arbitrary ultrasound image information. For example, training images 210 may include image information used to train the ultrasound image recognition module 120, such as various ultrasound image information associated with known views of organs such as the heart. As a further example, training images 210 may be clinically desirable images, such as a view of the heart over the sternum. In such cases, training images 210 may be ultrasound images predetermined (e.g., by a physician) as appropriately representing a clinically desirable view of the heart over the sternum. Each such training image 210 may have slightly different characteristics (e.g., higher quality image, lower quality image, blurry image, image taken at a slightly different angle, etc.), yet each such training image 210 may be predetermined as adequately representing a clinically desirable view of the heart or other anatomical structure.

[0034] Furthermore, the training images 210 may include not only image information associated with clinically standard or desirable views, but also image information associated with clinically non-standard or undesirable views. Therefore, the ultrasound recognition module 120 may receive views of the heart that do not represent any particular clinically desirable view (e.g., suprasternal, subcostal, short-axis and long-axis parasternal, 2-ventricular apex, 3-ventricular apex, 4-ventricular apex, and 5-ventricular apex views). In such cases, the ultrasound recognition module 120 may nevertheless be trained to recognize the image as a view of the heart, and further, it may recognize the image as being somewhere between, for example, a 2-ventricular apex view and a 3-ventricular apex view. A clinically standard 3-ventricular apex view can generally be obtained, for example, by rotating the ultrasound imaging probe counterclockwise by about 60 degrees relative to a 2-ventricular apex view. An ultrasound image obtained using the probe with a rotation angle of, for example, 5° to 55° counterclockwise relative to a 2-ventricular apex view may be determined not to represent a clinically desirable view of the heart. The ultrasound image recognition module 120 may be trained with training images 210 that show various known but clinically undesirable views of the heart (such as a view somewhere between a two-ventricular apex view and a three-ventricular apex view), and thus be able to recognize such views (for example, the ultrasound image recognition module 120 may recognize a view as representing a 35° counterclockwise rotation of the probe 118 relative to a two-ventricular apex view). In some embodiments, when the ultrasound image is recognized to contain a known clinically undesirable view, guidance can be provided to the user to move the ultrasound probe in a way that ultimately achieves the acquisition of a clinically desirable view.

[0035] In some embodiments, the training images 210 may include image information used to train the anatomical structure recognition and labeling module 130. For example, the training images 210 may include various ultrasound image information associated with a known anatomical structure, such as a specific organ (e.g., the heart) or a specific feature of an organ (e.g., left ventricle, right ventricle, left atrium, right atrium, mitral valve, tricuspid valve, aortic valve, etc.). Furthermore, the training images 210 may include image information associated with such known anatomical structures from various different views. Anatomical structures can look very different in different views; for example, the left ventricle may look different in ultrasound images acquired in various different views (e.g., apical-LV, parasternal-LV, parasternal-LV). Therefore, ultrasound images representing a known anatomical structure (e.g., left ventricle) in various different views may be provided as training images 210 that can be used to train the anatomical structure recognition and labeling module 130 to recognize not only the anatomical structure but also the specific view provided by the ultrasound images.

[0036] In some embodiments, the training images 210 may include image information used to train the ultrasound image grading module 140. For example, the training images 210 may include various ultrasound images with different image quality (e.g., higher quality images, lower quality images, blurry images, etc.). The quality of the training images 210 used to train the ultrasound image grading module 140 may be graded by a specialist, such as a physician or other clinician. The quality of the training images 210 may be graded based on any grading system. In some embodiments, the quality of the training images 210 may be graded based on a standard grading system, such as the American College of Emergency Physicians (ACEP) grading rubric provided in Table 1 below. Each of the training images 210 may be assigned a specific grade (e.g., 1-5) by a physician or other clinician, and the assigned grade represents the quality of the training image 210.

[0037] [Table 1]

[0038] Other training inputs 220 may be further provided to the ultrasound image recognition module 120 for training. Other training inputs 220 may include, for example, manually entered inputs to adjust or otherwise manage the image recognition model developed in the image recognition module 120 throughout the training process.

[0039] Using the training images 210, the machine learning circuit 105 (including the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140) can perform an iterative training process. Training can be based on a wide variety of learning rules or training algorithms. For example, the learning rules may include one or more of the following: backpropagation, real-time repetition learning, pattern unit learning, supervised learning, interpolation, weighted sum, reinforcement learning, lag learning, unsupervised learning, and / or recording learning.

[0040] The backpropagation learning algorithm is an example of a method for training an artificial neural network, which can be used, for example, in the artificial neural network 300 shown in Figure 3. Backpropagation generally involves two stages: propagation and weight updating. In the propagation stage, the input of the training pattern is forward propagated through the neural network to generate the output activations of the propagation. Next, the output activations of the propagation are backpropagated through the neural network using the training pattern target to generate the deltas (i.e., the difference between the input value and the output value) of all output neurons and hidden neurons. In the weight updating stage, for each weight-synapse, the following steps are generally performed: 1. Multiply its output delta by its input activation to obtain the gradient of the weight. 2. Subtract the ratio (percentage) of the gradient from the weight. The propagation and weight updating stages are repeated as needed until the network performance is satisfactory.

[0041] As a result of training, the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140 learn to modify their behavior in response to the training image 210 and can obtain or generate ultrasound image knowledge 230. The ultrasound image knowledge 230 can represent any information that enables the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140 to determine an appropriate response to new data or situations. For example, the ultrasound image knowledge 230 can represent a relationship between an ultrasound image and one or more views of an organ (e.g., ultrasound image parameters, coefficients, weighting information, parameters related to the exemplary neural network shown in Figure 3, or one or more functions that describe one or more views of an organ based on any such variables), which can be utilized by the ultrasound image recognition module 120 to recognize the ultrasound image. Furthermore, the ultrasound image knowledge 230 can represent a relationship between received ultrasound image information from various different views and the recognized anatomical structures represented in the received ultrasound image information. Furthermore, the ultrasound image knowledge 230 can represent the relationship between the received ultrasound image information and the image quality of the received ultrasound image information.

[0042] Ultrasound image knowledge 230 can be stored in the ultrasound image knowledge database 122.

[0043] Based on the training images 210, the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140 can learn to modify their behavior and apply the knowledge contained in the image knowledge database 122 to change how these modules make decisions regarding new inputs, such as ultrasound image information received from the ultrasound imaging device 110.

[0044] Figure 3 is a block diagram showing an example of an artificial neural network 300 that may be implemented by a machine learning circuit 105 in one or more embodiments. In some embodiments, each of the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140 may be implemented by a neural network such as the neural network 300 shown in Figure 3. An artificial neural network (ANN) is an artificial intelligence model that is generally unknown and is used to estimate or approximate functions that may depend on a large number of inputs. Such a neural network generally includes a system of interconnected "neurons" that exchange information with each other. The connections have numerical weights that can be adjusted based on experience, so the neural network is adaptive to inputs and has the ability to learn.

[0045] The artificial neural network 300 shown in Figure 3 includes three layers: an input layer 310 containing input neurons i1-i3, a hidden layer 320 containing hidden layer neurons h1-h4, and an output layer 330 containing output neurons f1 and f2. Although the neural network 300 in Figure 3 is shown to have three layers, it should be readily understood that additional layers can be included in the neural network 300 as desired to achieve optimal training and performance of the machine learning circuit 105. Similarly, the neurons in each layer are shown for illustrative purposes, and it should be readily understood that each layer may contain more, or even significantly more, neurons than those shown in Figure 3.

[0046] The neural network 300 can be trained by providing training images 210 to the input layer 310. As illustrated with reference to Figure 2, the training images may include ultrasound image information with a wide variety of known properties, such as various organ views, various known anatomical structures in various different imaging views, and various image quality or grades. Through training, the neural network 300 may generate and / or modify a hidden layer 320 that represents weight connections mapping the training images 210 provided in the input layer 310 to known output information in the output layer 330 (e.g., classification of an image as a specific imaging view of the heart, recognition of a specific anatomical structure in an image, classification of an image as having a specific image quality). The relationships between neurons in the input layer 310, hidden layer 320, and output layer 330, formed through the training process and which may include weight connection relationships, are generally referred to herein as “ultrasound image knowledge” and may be stored, for example, in an ultrasound image knowledge database 122.

[0047] Once the neural network 300 is sufficiently trained, it may be provided with an untrained ultrasound image in its input layer 310 (i.e., an ultrasound image taken from a patient using the ultrasound imaging device 110). Using the ultrasound image knowledge stored in the ultrasound image knowledge database 122 (which may include, for example, weight connection information between neurons in the neural network 300), the neural network 300 can make decisions about the received ultrasound image information in its output layer 330. For example, the neural network 300 can determine whether the received ultrasound image represents one or more clinically desirable or non-clinically desirable views of an organ, recognize one or more anatomical structures in the received ultrasound image, automatically label the recognized anatomical structures in the image, and even automatically determine and assign an image quality grade to the received ultrasound image.

[0048] The neural network 300 in Figure 3 is provided as merely one example of various possible implementations of the machine learning circuit 105, which employs artificial intelligence to make decisions regarding received ultrasound image information. For example, the machine learning circuit 105 (including one or more of the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140) can implement any of the following techniques for making decisions regarding ultrasound images received from a patient: neural networks, deep learning, convolutional neural networks, and Bayesian programmed learning techniques.

[0049] Furthermore, the ultrasound recognition module 120 may be trained to make various decisions related to the received ultrasound image information by utilizing various training images 210 and / or various sequences of training images 210. For example, the ultrasound recognition module 120 may be trained or otherwise configured to determine whether the received ultrasound image represents one or more clinically standard or desirable views. Furthermore, the ultrasound recognition module 120 may determine whether the received ultrasound image represents a non-clinically desirable view (and may recognize such a non-clinically desirable view as a specific view or angle of a particular organ or other tissue within the patient), and may further determine, based on the sequence of received ultrasound images, whether the image is moving towards or away from a clinically desirable view of the organ. For example, if the quality of the received ultrasound image improves from one image to another (for example, based on a quality rating provided by the ultrasound image grading module 140), the ultrasound recognition module 120 may determine that the user is moving towards obtaining a clinically desired view of the organ or other anatomical structure. Based on the recognition of whether the image is moving towards or away from a clinically desirable view of the organ, and / or based on the recognition of the image that has actually been captured, the system may then be configured to provide feedback to the user, for example, by indicating the direction in which the user may want to move the probe and / or the angle of rotation or orientation in which the user may want to angle the probe, in order to help the user capture the desired view of the organ.

[0050] For example, as discussed above, the ultrasound image recognition module 120 may be trained with training images 210 showing various known but clinically undesirable views of the heart (such as a view somewhere between the two-ventricular apex view and the three-ventricular apex view), and thus be able to recognize such views (for example, the ultrasound image recognition module 120 may recognize a view as representing a 35° counterclockwise rotation of the probe 118 relative to the two-ventricular apex view). Furthermore, the ultrasound image recognition module 120 may be trained with sequences of views of the heart that are recognized but are nonclinically standard or desirable. For example, the ultrasound image recognition module 120 may be trained to recognize ultrasound images showing views of the heart at each degree of counterclockwise rotation between 0° and 60° relative to the two-ventricular apex view (i.e., each degree between the two-ventricular apex view and the three-ventricular apex view). Furthermore, the ultrasound image recognition module 120 may be trained to recognize sequences or progressions of non-clinically desired views that move toward and / or away from clinically desired views (for example, training images 210 may include a sequence of ultrasound images representing the rotation of probe 118 toward and / or away from a two-ventricular apex view). Thus, the ultrasound image recognition module 120 may be trained to recognize that the received ultrasound images may not represent a specific clinically desired view, but may be continuously moving toward (or away from) a clinically desired view.

[0051] Furthermore, the ultrasound image recognition module 120 can be trained to determine whether the received ultrasound image represents one of several clinically desirable views of an organ. Such clinically desirable views of an organ may include, for example, suprasternal, subcostal, parasternal in the short and long axes, and 2-ventricular, 3-ventricular, 4-ventricular, and 5-ventricular apex views of the heart.

[0052] Referring again to Figure 1, the machine learning circuit 105 (including any of the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140) can provide feedback signals to the ultrasound imaging device 110 (for example, shown in reference to reference number 103) based on the analysis of the ultrasound images received by the machine learning circuit 105, as will be described in more detail below.

[0053] Figure 4 schematically shows an ultrasonic imaging device 110 according to one or more embodiments. The ultrasonic imaging device 110 may include a display 112, a user interface 410 including one or more input elements 412, one or more visual feedback elements 420, an audible feedback element 430 and / or a tactile feedback element 440.

[0054] The user interface 410 allows the user to control or otherwise communicate with the ultrasound imaging device 110. Various types of user input can be provided via user input elements 412, which may be, for example, buttons or similar user input elements. Furthermore, or alternatively, the display 112 may be a touchscreen display, and user input may be received via the display 112. Using the ultrasound imaging device 110, the user can select or otherwise input a desired view of an organ to be imaged in a patient (e.g., via input elements 412 and / or the display 112). For example, the user can select one view (e.g., a subcostal view of the heart) from several clinically desirable views stored in the ultrasound imaging device 110 and presented to the user. The ultrasound imaging device 110 may transmit the selected view to the ultrasound image recognition module 120, which may be configured to determine whether the received ultrasound image represents the selected view. In other words, the ultrasound image recognition module 120 can access appropriate ultrasound image knowledge in the image knowledge database 122 (e.g., knowledge, rules, or relationships associated with a subcostal view of the heart) so that the received ultrasound image can be compared to or processed by knowledge corresponding to a selected view. Alternatively, the user can select an operating mode in which the system guides the user through the acquisition of one or more standard views of an organ such as the heart as described above. In such a mode, the system can first select a desired view of the organ to be imaged, then confirm with the user when the desired image is acquired, and / or guide the user toward the desired view based on the initial image acquisition. For example, if the ultrasound image recognition module 120 determines that the received ultrasound image represents a particular selected view of an organ, the system 100 can provide the user with instructions (e.g., visual, auditory, or tactile feedback) to confirm the acquisition of the selected view (e.g., via the ultrasound imaging device 110).On the other hand, if the ultrasound image recognition module 120 determines that the received ultrasound image does not represent a specific selected view of the organ, the system 100 may provide the user with instructions (e.g., visual, auditory, or tactile feedback) to guide the user toward acquiring the selected view, such as providing instructions for user actions of the probe 118 to acquire the selected view. Instructions for user actions of the probe 118 may include, for example, instructions for a specific direction and / or amount of rotational or translational movement of the probe 118 to acquire the selected view.

[0055] The system 100 can then sequentially repeat this process for each of the desired standard views of the organ to be imaged. That is, for each of the series of views of the organ to be acquired, the system 100 can iteratively guide the user to acquire a specific selected view among the series of views and confirm when each selected view has been acquired. Alternatively, in some embodiments, the system 100 is configured to operate in such a way that it compares any acquired image with each of the images to be taken, without initially instructing which view to acquire first, and confirms when one or more of the desired standard views have been acquired. For example, in some embodiments, it is not necessary to select a specific view first. Instead, the system 100 can automatically recognize when a specific desired view (e.g., a clinically standard view of the heart) has been acquired during the imaging session. Once a specific desired view has been acquired, the system 100 can automatically provide the user with an instruction (e.g., via the ultrasound imaging device 110) confirming that the desired view has been acquired, allowing the user to proceed with the patient's examination. Similarly, when a second desired view is acquired, the system 100 can automatically provide the user with instructions to confirm that the second view has been acquired. In various embodiments, the system 100 can automatically store the ultrasound images representing the desired view after they have been captured, for example, by storing the ultrasound images in the ultrasound image database 115.

[0056] The visual feedback element 420 may be any element that can provide a visual display to the user of the ultrasonic imaging device 110, and may be, for example, one or more lights, colors, shapes, icons, etc., whether static or dynamic. The audible feedback element 430 may be any element that can generate audible instructions to the user of the ultrasonic imaging device 110, and may be, for example, a speaker for generating the lack of a correspondence between a captured image and an image to be captured, and various tones or sounds associated with the correspondence. Similarly, the tactile feedback element 440 may be any element that can provide a tactile effect to the user of the ultrasonic imaging device 110, and may be, for example, a vibration device.

[0057] The feedback signal 103 provided by the ultrasound image recognition module 120 may indicate one of the various decisions made by the ultrasound image recognition module 120 regarding the ultrasound image received from the ultrasound imaging device 110.

[0058] For example, the ultrasound image recognition module 120 may provide a feedback signal 103 indicating that the current or most recently received ultrasound image represents a clinically desirable view of the organ (e.g., a selected clinically desirable view). In a further example, the ultrasound image recognition module 120 may determine whether the received ultrasound image is continuously moving towards or away from a clinically desirable view of the organ and provide a feedback signal 103 indicating whether the received ultrasound image is continuously moving towards or away from a clinically desirable view of the organ, for example, based on an increase or decrease in the ultrasound image quality determination by the ultrasound image grading module 140, or based on a sequence of recognized images or structures known to correspond to the progression of images indicating movement toward or away from a clinically desirable view of the organ. This feedback signal may include visual or auditory commands instructing the probe to move or angle in a certain direction, or icons such as straight or curved arrows(s) indicating the direction and / or angle of movement required for the probe to move closer to a desired image of the organ.

[0059] The ultrasound imaging device 110 may receive a feedback signal 103 and, in response, activate one or more feedback elements (i.e., a visual feedback element 420, an audible feedback element 430, and / or a tactile feedback element 440) to provide a feedback effect to the user of the ultrasound imaging device 110. For example, the feedback signal 103 may indicate that the currently or recently received ultrasound image represents a clinically desirable view of an organ. In such a case, the feedback effect provided by the ultrasound imaging device 110 may include flashing green light 420a from the visual feedback element 420, an audible tone or beep from the audible feedback element 430, and / or vibration pulses provided by the tactile feedback element 440. The flashing green light 420a, audible sound, and / or vibration pulses indicate to the user that the desired view has been obtained, and the user can therefore retain the ultrasound image of the desired view (e.g., using one or more of the user input elements 412) and store the image in the ultrasound image database 115.

[0060] Furthermore, or alternatively, if it determines that a clinically desirable view of an organ is represented in the received ultrasound image, the ultrasound image recognition module 120 can cause the ultrasound imaging device 110 to automatically retain the ultrasound image and store it in the ultrasound image database 115 (e.g., by a feedback signal 103). Next to each desired type of image, a table may also be displayed with appropriate instructions indicating whether the user has already captured the desired image or whether there remains a need to capture the desired image for a particular patient during imaging. In some embodiments, the table includes a set of ultrasound images obtained in relation to a particular type of ultrasound imaging session. The set of ultrasound images may include several different views acquired during a particular type of ultrasound imaging session (e.g., a cardiac imaging session), such as one or more of the following: suprasternal, subcostal, short-axis and long-axis parasternal, 2-ventricular apex, 3-ventricular apex, 4-ventricular apex, and 5-ventricular apex views of the heart. The table may contain various different types of ultrasound imaging sessions, such as ultrasound imaging sessions or examinations for specific pathologies (e.g., lung pathology, cardiac pathology, etc.), specific anatomical structures (e.g., lungs, heart, etc.), or any other type of ultrasound imaging session or examination. Each ultrasound imaging session may have an associated set of desired or clinically standard views to be obtained, and each such view may be included in the table for that particular ultrasound imaging session. When each of the desired or clinically standard sets of views is obtained, an entry may be automatically created in the table to indicate the acquisition of such views.

[0061] In embodiments in which the feedback signal 103 indicates that the received ultrasound image is continuously approaching or moving away from a clinically desired view of an organ, the ultrasound imaging device 110 can communicate this to the user by providing a changing feedback effect, such as an audible sound whose frequency increases (or decreases) as the received ultrasound image approaches (or moves away from) a clinically desired view, a series of vibration pulses whose intensity increases (or decreases) as the received ultrasound image approaches (or moves away from) a clinically desired view, and / or illuminating the received ultrasound image with light of a different color or position as it approaches (or moves away from) a clinically desired view (for example, illuminating the received ultrasound image with red outer light 420c, then yellow middle light 420b, then green central light 420a as it approaches a clinically desired view).

[0062] In some embodiments, the feedback signal 103 may represent information derived from or provided by the ultrasound image grading module 140. For example, the feedback signal 103 may indicate a predicted or determined grade (e.g., grades 1 to 5) of the acquired ultrasound image, which may be provided to the user, for example, by displaying the grade together with the ultrasound image displayed on the display 112, or by audible, visual, tactile, or other feedback mechanisms. In some embodiments, the image quality grade that may be represented by the feedback signal 103 may be used to guide the user to acquire a clinically desirable ultrasound image. For example, the feedback signal 103 may indicate that the acquired ultrasound image is of insufficient or non-clinically useful quality, and the user may adjust or reposition the probe 118 until an ultrasound image of adequate quality is obtained.

[0063] In some embodiments, the predicted or determined grade of an acquired ultrasound image, which is determined to represent a specific view of an anatomy, structure, or organ, or otherwise identified, may be automatically associated with the specific view and stored, for example, in a table.

[0064] In some embodiments, the feedback signal 103 can represent information provided by the anatomical structure recognition and labeling module 130 or the ultrasound image grading module 140 in response to the analysis of the received ultrasound image by the anatomical structure recognition and labeling module 130 or the ultrasound image grading module 140. For example, the anatomical structure recognition and labeling module 130 may recognize anatomical structures in the received ultrasound image and automatically provide labels for the recognized anatomical structures, which may be displayed together with the ultrasound image, for example, on the display 112.

[0065] Figure 5 shows an ultrasound image 500 including labels automatically associated by the anatomical structure recognition and labeling module 130, and is a figure containing anatomical structures that are recognized by the anatomical structure recognition and labeling module 130. The anatomical structure recognition and labeling module 130 is configured (e.g., through training) to recognize anatomical structures in the received ultrasound image and to localize the recognized structures, i.e., to recognize or determine the location of the recognized anatomical structures in the ultrasound image. Training of the anatomical structure recognition and labeling module 130 may be performed as described herein, for example, using previously acquired ultrasound images in which anatomical structures have been identified, localized, and labeled in the image by human expert interpretation of the ultrasound image. In this way, the anatomical structure recognition and labeling module 130 can determine the correct location for the label, i.e., the location corresponding to the determined location of the recognized anatomical structure in the ultrasound image. The labels shown in Figure 5 include labels for the right ventricle (RV), left ventricle (LV), tricuspid valve (TV), mitral valve (MV), right atrium (RA), and left atrium (LA). These labels are displayed at the location in the displayed ultrasound image corresponding to the structure identified by the label. The labels shown in Figure 5 are provided as some examples of structures that can be automatically recognized and labeled by the anatomical structure recognition and labeling module 130. However, embodiments of this disclosure are not limited thereto, and in various embodiments, labels associated with any anatomical structure may be automatically determined and displayed on the ultrasound image.

[0066] As the view of the ultrasound image changes, for example, due to the movement of the probe 118 and / or the patient being imaged, the labels within the ultrasound image 500 may also change. For example, if a recognized anatomical structure appears or disappears from the displayed ultrasound image, the label corresponding to the recognized anatomical structure will appear or disappear accordingly. As another example, if an anatomical structure in a sequence of displayed ultrasound images is recognized as moving within the displayed image, the anatomical structure recognition and labeling module 130 may dynamically rearrange the labels within the image as the recognized anatomical structure moves within the image.

[0067] Furthermore, in some embodiments, the output of the anatomical structure recognition and labeling module 130 is temporally smoothed in the video stream of the acquired ultrasound image data. For example, the results of the analysis by the anatomical structure recognition and labeling module 130 (e.g., recognition and labeling of anatomical structures) may be stored in a circular buffer. In some embodiments, the displayed results (e.g., recognition and labeling of structural structures) represent a calculation of the geometric mean of the results in the buffer. In this way, the influence of outliers in the decisions made in anatomical structure recognition and labeling (presence and / or location of anatomical structures) may be reduced, and the movement of displayed labels in the image may be smoothed to reduce jitter or similar effects caused by probe movement 118, movement of anatomical structures (e.g., cardiac contraction / expansion), etc.

[0068] In some embodiments, the labels that may be associated with a recognized anatomical structure may be limited based on the view of the acquired ultrasound image. For example, in some embodiments, the anatomical structure recognition and labeling module 130 may not only recognize the anatomical structure represented in the acquired ultrasound image, but may also recognize the view from which the ultrasound image is obtained (e.g., apex-LV, parasternal-LV, parasternal-LV). As previously stated herein, anatomical structures may look very different in various different ultrasound imaging views. Anatomical structures in various different views may be treated, for example, as separate classes for recognition by the anatomical structure recognition and labeling module 130. Thus, the output generated by the anatomical structure recognition and labeling module 130 may be limited in terms of the labels that may be attached to or otherwise associated with the recognized anatomical structure, depending on the view of the ultrasound image. In some embodiments, once a particular view of a particular anatomical structure is determined, the anatomical structure recognition and labeling module 130 may determine a label to associate with the recognized anatomical structure, and the determined label may be limited to a label within a specific set of labels associated with the particular view of the anatomical structure.

[0069] Figures 6A and 6B show ultrasound images including image grades that are automatically associated with the ultrasound image by the ultrasound image grading module 140.

[0070] Figure 6A shows an ultrasound image 601 automatically graded as image quality "1," which in some embodiments may represent a low-quality image. In some embodiments, as discussed earlier herein, a standard grading system such as the ACEP grading rubric (see Table 1) may be used or implemented by the ultrasound image grading module 140 when grading received ultrasound images. An image quality of "1" depicted in ultrasound image 601 may represent an ultrasound image that lacks recognizable structures and from which objective data cannot be collected, which corresponds to the standard grade or score of 1 in the ACEP image quality grading rubric. In some embodiments, the ultrasound image grading module 140 is configured to annotate ultrasound images graded with an automatically determined grade. For example, as shown in Figure 6A, ultrasound image 601 includes a label "1" indicating the image grade of ultrasound image 601 as determined by the ultrasound image grading module 140. For example, any appropriate identifying information can be used to display the determined image grade of an ultrasound image, including numbers, text descriptions, and colors (for example, red may indicate poor image quality, and green may indicate good image quality).

[0071] Figure 6B shows an ultrasound image 602 that has been automatically graded as image quality "3," which in some embodiments may represent an image of intermediate quality. In some embodiments, as discussed earlier herein, a standard grading system such as the ACEP grading rubric (see Table 1) may be used or implemented by the ultrasound image grading module 140 when grading the received ultrasound images. An image quality of "3" depicted in ultrasound image 601 represents an ultrasound image that meets the minimum criteria for diagnostic suitability of the ultrasound image, where the structure is recognizable, but has some technical or other defects, which corresponds to the standard of grade or score 3 in the ACEP image quality grading rubric.

[0072] In some embodiments, the user may be guided toward acquiring a high-quality ultrasound image (and, in some embodiments, toward acquiring an image representing a clinically desirable view) through the use of the displayed image quality grade. For example, if the ultrasound image is displayed as quality "1", the user may slowly move or reposition the probe, adjust the probe's imaging parameters (e.g., depth, gain, etc.), etc. As the user adjusts the probe, the image quality may improve, and this is used as feedback to the user to indicate that they are approaching a higher quality image.

[0073] Although the machine learning circuit 105 (including the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140) has been described herein as being separate from the ultrasound imaging device 110 and accessible via the communication network 102, it should be readily understood that the machine learning circuit 105 may be contained within the ultrasound imaging device 110. That is, the machine learning circuit 105 (including the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140) may be contained within the ultrasound imaging device 110, for example, stored in memory 114, and the features and / or functions of the machine learning circuit 105 may be executed or otherwise implemented by the processor 116.

[0074] In some embodiments, one or more of the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140 may be implemented by a single neural network optimized for real-time performance on a mobile device. For example, one or more of the ultrasound image recognition module 120, the anatomical structure recognition and labeling module 130, and the ultrasound image grading module 140 may be implemented by a single neural network executed by or stored in an ultrasound imaging device 110, which may be a mobile device such as a laptop or tablet computer or a smartphone.

[0075] In some embodiments, the anatomical structure recognition and labeling module 130 may have inference time to recognize anatomical structures within each of the received ultrasound images within a time interval such that processing of the received ultrasound images is completed before the next ultrasound image becomes available (e.g., received and available for processing by the anatomical structure recognition and labeling module) during real-time ultrasound imaging. In some embodiments, the anatomical structure recognition and labeling module 130 may be configured to recognize anatomical structures (e.g., object detection) and specific ultrasound imaging views (e.g., view classification) within 25 milliseconds from the reception of the ultrasound image information. However, the embodiments provided herein are not limited thereto, and in some embodiments, the anatomical structure recognition and labeling module 130 is configured to recognize anatomical structures and ultrasound imaging views in ultrasound images in a time of less than or greater than 25 milliseconds.

[0076] This application claims priority to U.S. Provisional Patent Application No. 62 / 899,554, filed on 12 September 2019, which is incorporated herein by reference in its entirety.

[0077] Further embodiments can be provided by combining the various embodiments described above. These and other modifications can be made to embodiments, taking into consideration the modes for carrying out the above invention. In general, the terms used in the following claims should not be interpreted as limiting the claims to the specification and the specific embodiments disclosed herein, but rather as including all possible embodiments within the scope of all equivalents to which such claims are granted. Accordingly, the claims are not limited by this disclosure.

Claims

1. An ultrasound imaging device configured to acquire ultrasound images of a patient, The system comprises an anatomical structure recognition and labeling circuit at least partially implemented by a mechanical learning circuit including at least one artificial neural network, wherein the anatomical structure recognition and labeling circuit is operable according to executable instructions stored in memory, and when the executable instructions are executed by one or more processors, the anatomical structure recognition and labeling circuit Receiving the acquired ultrasound image from the ultrasound imaging device, The view of the received ultrasound image is automatically recognized by the mechanical learning circuit. The mechanical learning circuit automatically recognizes one or more anatomical structures within the received ultrasound image. Based on the view recognized by the mechanical learning circuit, the system automatically accesses from memory a predetermined set of labels previously stored in memory in relation to the view, where the labels in the predetermined set of labels identify different anatomical structures. Using the output from the mechanical learning circuit, a label for an anatomical structure in the received ultrasound image is automatically selected from a predetermined set of labels, wherein the output from the mechanical learning circuit indicates the anatomical structure, and the automatic selection of the label is limited to a predetermined set of labels, and The system is configured to automatically correct one or more of the received ultrasound images and label the anatomical structures with labels selected from a predetermined set of labels. Furthermore, an ultrasound system comprising a display configured to display one or more of the received ultrasound images having the labeled anatomical structures.

2. The ultrasound system according to claim 1, wherein the display is configured to display information identifying one or more anatomical structures at a location in the received ultrasound image corresponding to the location of the anatomical structure.

3. The ultrasound system according to claim 1, wherein the anatomical structure recognition and labeling circuit is configured to automatically recognize each of the views of the received ultrasound images before the next of the ultrasound images is received from the ultrasound imaging device.

4. The aforementioned anatomical structure recognition and labeling circuit, To determine the placement of the labels for the anatomical structures in each ultrasound image in multiple sequentially acquired ultrasound images, The ultrasound system according to claim 1, further configured to temporally smooth the output of the label in the continuously acquired ultrasound images based on at least one of the determined presence or determined location of the anatomical structure in one or more previous ultrasound images accompanying the current ultrasound image in the continuously acquired ultrasound images.

5. An ultrasonic image grading circuit, The ultrasonic imaging device receives the acquired ultrasonic image, The image quality of the received ultrasound image is automatically graded, The ultrasonic system according to claim 1, further comprising an ultrasonic image grading circuit configured to assign an image quality grade to the received ultrasonic image.

6. The ultrasonic system according to claim 5, wherein the display is configured to display an indication of the image quality grade of the received ultrasonic image.

7. The ultrasonic system according to claim 6, wherein the indication of the image quality grade includes at least one of a number indicating the image quality grade, a text description indicating the image quality grade, or a color indicating the image quality grade.

8. The ultrasonic system according to claim 6, wherein the indication for the image quality grade is an integer from 1 to 5.

9. The ultrasonic image grading circuit, The ultrasound system according to claim 5, further configured to provide feedback to the user based on the image quality grade, wherein the feedback is configured to guide the user to obtain the selected view of the anatomical structure by providing the user with a display of user actions on the ultrasound imaging device to obtain the selected view.

10. The ultrasonic system according to claim 5, wherein the ultrasonic image grading circuit is at least partially implemented by a machine learning circuit including at least one artificial neural network.

11. A method for automatically labeling anatomical structures in ultrasound images, An anatomical structure recognition and labeling circuit receives ultrasound images acquired by an ultrasound imaging device, wherein the anatomical structure recognition and labeling circuit is at least partially implemented by a mechanical learning circuit, and the mechanical learning circuit includes at least one artificial neural network. The mechanical learning circuit automatically recognizes the view of the received ultrasound image, The mechanical learning circuit automatically recognizes one or more anatomical structures in the received ultrasound image, Based on the view recognized by the mechanical learning circuit, the system automatically accesses a predetermined set of labels previously stored in memory in relation to the view, where the labels in the predetermined set of labels identify different anatomical structures. The anatomical structure recognition and labeling circuit automatically labels the anatomical structures in the received ultrasound image with information that identifies the anatomical structures, and here the automatic labeling is performed by: Using the output from the mechanical learning circuit, a label for the anatomical structure is automatically selected from a predetermined set of labels, wherein the output from the mechanical learning circuit represents the anatomical structure, and the automatic selection of the label is constrained to a predetermined set of labels. Automatically correcting one or more of the received ultrasound images and labeling the anatomical structures with the labels selected from the predetermined set of labels, and A method comprising displaying one or more of the received ultrasound images having the labeled anatomical structures.

12. The method according to claim 11, wherein displaying one or more of the received ultrasound images having the labeled anatomical structures includes displaying information that identifies the anatomical structures at a location in the displayed ultrasound image corresponding to the location of the anatomical structures.

13. The method according to claim 11, further comprising: automatically grading the image quality of the received ultrasound image using an ultrasound image grading circuit; and assigning the image quality grade to the received ultrasound image.

14. The method according to claim 13, further comprising displaying an indication of the image quality grade of the received ultrasound image.

15. It is an ultrasonic system. An ultrasound imaging device configured to acquire a series of ultrasound images of a patient; and It includes an anatomical structure recognition and labeling circuit capable of operating according to executable instructions stored in memory, wherein when the executable instructions are executed by one or more processors, the anatomical structure recognition and labeling circuit is configured to do the following: Receiving the ultrasound images within the series of ultrasound images from the ultrasound imaging device; Viewing the ultrasound image and automatically recognizing one or more anatomical structures in the ultrasound image; Based on the recognized view, access from memory a predetermined set of labels previously stored in memory in relation to the view, where the labels in the predetermined set of labels identify different anatomical structures; Selecting labels for anatomical structures in the ultrasound image from a predetermined set of labels, wherein the selection of labels is constrained to the predetermined set of labels; In the ultrasound image, determining the placement of the label for the anatomical structure; Smoothing out the output of the label in the ultrasound image in time based on at least one determined presence or determined location of the anatomical structure in one or more previous ultrasound images having the current ultrasound image in the series of ultrasound images; and Automatically correct the ultrasound image and label the anatomical structures with labels selected from a predetermined set of labels in the time-smoothed arrangement. Furthermore, the ultrasound system includes a display configured to show the ultrasound images having labeled anatomical structures. Ultrasonic system.

16. An ultrasound system according to claim 15, wherein the display is configured to display a series of ultrasound images modified with labels that identify the anatomical structures in the temporally smoothed arrangement. Ultrasonic system.

17. An ultrasound system according to claim 15, wherein the anatomical structure recognition and labeling circuit is configured to automatically recognize the view for each ultrasound image in a series of ultrasound images before receiving one of the ultrasound images from the ultrasound imaging device. Ultrasonic system.

18. The ultrasonic system according to claim 15, further comprising an ultrasonic image grading circuit capable of operating in accordance with the executable instructions in the memory, wherein the execution of the executable instructions by one or more processors causes the ultrasonic image grading circuit to perform the following: Automatically grade the image quality of the aforementioned ultrasound images; Assigning image quality grades to the ultrasound images; and Automatically correct the ultrasound image and include the display of the image quality grade. Ultrasonic system.

Images