Apparatus and method for automatic ultrasonic segmentation for visualization and measurement

Abstract

To further efficiently acquire image data from human subjects using ultrasonic device.SOLUTION: A system and method for performing ultrasonic scans is provided. An ultrasonagraphic system acquires sonogram information from a series of ultrasonic scans of a human subject. The series of ultrasonic scans are taken over a portion of interest on the human subject having an underlying bone structure or other ultrasonically discernible organ that is under examination. The data from the series of scans are synthesized into a single data file that corresponds to a three-dimensional (3D) image and / or 3D model of the underlying bone structure or organ of the human subject to be examined.

Description

Technical Field

[0001] This application claims the benefit of priority of U.S. Patent Application No. 16 / 813,469, filed Mar. 9, 2020, which is hereby incorporated by reference in its entirety for all purposes.

Background Art

[0002] In the technical field of visualizing the human body, particularly the human spine, X-ray images and computed tomography (CT) scans have been fairly effective in obtaining image data of human body parts, particularly human bone structures such as the spine. Magnetic resonance imaging (MRI) is another tool for obtaining image data of human body parts.

[0003] However, X-ray devices, MRI devices, and CT scan devices are very expensive to acquire and operate. X-ray images present graphical information on a limited two-dimensional plane. MRI is unacceptably slow and provides low-resolution images of bone structures.

[0004] Furthermore, X-ray imaging and CT scans use ionizing radiation (X-rays) that can be harmful to human subjects, particularly when human subjects have to undergo repeated examinations over a long period of time. For example, a human subject suffering from progressive scoliosis (curvature of the spine) may have to be examined to confirm the degree and / or changes in the curvature of the spine. Repeated radiation exposure during regular examinations can be harmful to such human subjects.

[0005] To obtain information about a human subject, other less harmful devices are available. For example, an ultrasonic device projects sound waves into the body of a human subject and detects the returning sound wave echoes to generate an image called a sonogram. The ultrasonic device used in an ultrasonic diagnostic system generates sound waves at a frequency higher than about 20 kHz, which is the audible range of human hearing. Sound waves in the range of 2 - 18 MHz are often used for ultrasonic medical diagnostic purposes. Currently, no long-term side effects are known from examining the human body with ultrasound.

[0006] However, an ultrasound scan can cover only a relatively small part of a human subject's body per scan. Furthermore, a sonogram is a relatively narrow image that covers a relatively small cross-section of only about 5 cm - 7.5 cm (2 - 3 inches). Also, the objects identified in a sonogram are often blurry. For example, to obtain a sufficient amount of image data for the analysis of an entire human spine, 500 - 1000 sonogram images have to be captured. Therefore, an old-fashioned ultrasound scanner is insufficient for obtaining image information of a human subject's body when it has to examine a large area of a human subject, such as the spine, because the sonogram images are too small and it is not possible to easily analyze a large number of sonogram images to reach significant information about the condition of the human subject being examined.

[0007] Therefore, there is a need in the art to more effectively obtain image data from a human subject using an ultrasonic device. SUMMARY OF THE INVENTION

[0008] Embodiments of an ultrasonic diagnostic system provide a system and method for performing ultrasonic scanning. One embodiment of the ultrasonic diagnostic system obtains sonogram information from a series of ultrasonic scans of a human subject. The series of ultrasonic scans is performed across a target portion of a human subject having a subjacent bone structure or other ultrasonically distinguishable organ during an examination. Data from the series of scans is combined into a single data file corresponding to a three-dimensional (3D) image and / or 3D model of the subjacent bone structure or organ of the human subject being examined.

[0009] The components in the drawings are not necessarily drawn to scale relative to each other. Like reference numerals indicate corresponding parts throughout several views.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Modes for Carrying Out the Invention

[0011] An embodiment of the ultrasonic diagnostic system 100 provides a system and method for obtaining sonogram information from a series of ultrasonic scans of a human subject. The series of ultrasonic scans is performed across a target portion of a human subject having an underlying bone structure or other ultrasonically distinguishable organ during an examination. Data from the series of scans is synthesized into a single data file corresponding to a 3D image and / or 3D model of the underlying bone structure or organ of the human subject being examined.

[0012] The disclosed ultrasonic diagnostic system 100 will be better understood by considering the following detailed description in conjunction with the drawings. The detailed description and the drawings provide merely examples of the various inventions described herein. Those skilled in the art will understand that the disclosed examples may be modified, corrected, and changed without departing from the scope of the inventions described herein. Many variations are possible for different applications and design considerations, but for the sake of brevity, every possible variation is not described individually in the following detailed description.

[0013] Throughout the following detailed description, examples of various ultrasonic diagnostic systems 100 are provided. Related features in the examples may be the same, similar, or different in different examples. For the sake of brevity, related features are not described repeatedly in each example. Instead, by using the name of the related feature, it will inform the reader that the feature having the name of the related feature may be similar to the related feature in the previously described example. Features specific to a given example will be described in that particular example. The reader should understand that a given feature need not be the same or similar to the specific description of the related feature in any given figure or example.

[0014] The following definitions apply to this specification unless otherwise indicated.

[0015] "Substantially" means generally conforming to the particular dimension, range, shape, concept, or other aspect modified by that term, and the feature or component need not conform exactly. For example, a "substantially cylindrical" object means that it resembles a cylinder but may have one or more deviations from a true cylinder.

[0016] The terms "comprising", "including", and "having" (and their conjugations) are used interchangeably to mean including but not necessarily limited to, and are open-ended terms not intended to exclude additional elements or method steps not explicitly recited.

[0017] Terms such as "first", "second", and "third" are used to distinguish or identify various members such as groups and are not intended to imply a sequential, chronological, or numerical limitation.

[0018] "Connected" means connected either permanently or removably, directly or indirectly through intervening components.

[0019] FIG. 1 is a schematic diagram of an ultrasonic diagnostic system 100 for acquiring 3D image information and 3D model data for the bone structure or other internal organs of a human subject 102. Various examples are described herein in terms of examining and treating the condition of the spinal column by acquiring 3D image information and 3D model data for the spinal column of a human subject 102. Alternatively, or additionally, two-dimensional (2D) image information and / or 2D model data can be generated by an embodiment of the ultrasonic diagnostic system 100.

[0020] For non-limiting example uses, the ultrasonic diagnostic system 100 of FIG. 1 is configured to enable an operator to obtain in real time an ultrasonic image (sonogram) of a patient's spine with an ultrasonic transducer probe 104. After receiving sonogram image information from a series of ultrasonic scans, the ultrasonic diagnostic processor system 106 generates 3D image information and / or 3D model data of the spine of the human subject 102 without exposing the patient to potentially harmful ionizing radiation. Further, with the ultrasonic diagnostic system 100 of FIG. 1, an operator can obtain an image of the outer cortex of the patient's spine with high resolution in real time or on a substantially real-time basis. Those skilled in the art will understand that the ultrasonic diagnostic processor system 106 can be used to generate 3D images and / or 3D model data for other parts of the human subject 102. In some examples, the system is optionally configured to display images three-dimensionally and stereoscopically, such as by using the 3D visualization module and 3D / 2D stereoscopic display 108 shown in FIG. 1.

[0021] An exemplary embodiment of the ultrasonic diagnostic processor system 106 includes an ultrasonic interface 110, an ultrasonic image data processor 112, at least one image capture device 114, an optical tracking unit 116, an image registration module 118, an image processing algorithm module 120, a 3D / 2D visualization module 122, and a database 124. Some embodiments include an optional clock 126. The ultrasonic interface 110 communicably couples the ultrasonic transducer probe 104 to the ultrasonic diagnostic processor system 106 via a wired or wireless connection 128.

[0022] In an alternative embodiment, the image registration module 118, the image processing algorithm module 120, and / or the 3D visualization module may be integrated and / or integrated with other logic. In other embodiments, some or all of these memories and other data manipulation functions may be provided by using a remote server or other electronic device properly connected via the Internet or otherwise a client device (not shown). The database 124 may be implemented using any suitable local and / or remote memory device or system. Depending on the embodiment, the database 124 may be a dedicated memory system, may be part of other components or systems, and / or may be a distributed local and / or remote memory system. The database 124 may also include other logic, modules and / or databases not illustrated or described herein. Other ultrasound diagnostic systems 100 may include some of the components described above or may omit some of them. Further, additional components not described herein may be included in alternative embodiments.

[0023] As conceptually shown in FIG. 1, the ultrasound diagnostic system 100 uses optical tracking technology to accurately detect the position of the scanned part of the patient relative to the position of the ultrasound transducer probe 104 in the 3D space. The image capture device 114 acquires image information in the space around the human subject 102 and the ultrasound transducer probe 104. The image information (alternatively referred to herein as a camera image) is preferably acquired continuously periodically at a rate of dozens or hundreds of image frames per second. The image capture device 114 may be any suitable device that periodically captures a still image or captures a video image (known in the art as a series of still images in time series). The captured image data is then communicated from the image capture device 114 to the optical tracking unit 116.

[0024] Each of the acquired camera images has a related time stamp that identifies the camera image capture time or the camera image acquisition time. The camera image capture time can be expressed in real time or using a reference time. The time stamp information is provided by an internal clock in one or more image capture devices 114. Alternatively, the clock 126 can add time stamp information to the acquired camera image when the acquired camera image is communicating from the image capture device 114 to the optical tracking unit 116.

[0025] In some embodiments, multiple image capture devices 114 may be used to capture camera images synchronously. That is, the multiple image capture devices 114 provide camera images captured simultaneously with the same time stamp.

[0026] The optical tracking unit 116 identifies one or more optical targets 130 disposed on the surface of the body of the human subject 102 for each acquired image. Also, the optical tracking unit 116 identifies one or more optical targets 132 disposed on the surface of the ultrasonic transducer probe 104 in each acquired image.

[0027] The optical targets 130, 132 may be conventional optical targets that can be identified by the optical tracking unit 116, specially developed optical targets, or optical targets developed later. In some embodiments, the optical targets 130, 132 extend three-dimensionally around three coordinate axes and include separate optical target portions representing each axis. In other embodiments, the optical targets 130, 132 extend three-dimensionally around six axes and include separate optical targets representing each of the six axes. The optical targets 130, 132 may be active, such as by emitting an infrared signal to the optical target, or passive, such as including a retroreflective marker fixed to some interaction device.

[0028] Next, the optical tracking unit 116 calculates or determines the position of the ultrasonic transducer probe 104 relative to the optical target 130 in 3D space for the indexed time. The determination of the position is based on the identified relative positions of the optical targets 130, 132 in the acquired camera images. One skilled in the art will understand that the relative position between the optical targets 130, 132 may be based on their identified positions in the image. Further, the orientations of the optical targets 130, 132 can be determined from the analysis of the images of the optical targets 130, 132. Thus, the position and orientation of the ultrasonic transducer probe 104 relative to the optical target 130 can be determined.

[0029] Next, the optical tracking unit 116 determines the corresponding position on the body of the human subject 102. The determined position on this human subject 102 is, interchangeably herein, referred to as time-indexed position information. The time-indexed position information identifies the position and time when the ultrasonic transducer probe 104 was on the human subject 102. Any suitable position tracking system known currently or developed later may be used by various embodiments of the ultrasonic diagnostic system 100 to determine the time-indexed position information.

[0030] Note that the ultrasonic diagnostic system 100 of FIG. 1 is configured to directly detect the position of the human subject 102 not only by detecting the position of a fixture near the human subject 102, such as a chest plate or other fixed reference object, but also by one or more optical targets 130 disposed on the human subject 102. Thus, if the human subject 102 moves or adjusts their position during the examination, the time-indexed position information determined from the later acquired camera images can be correlated with the time-indexed position information determined from the previously acquired camera images.

[0031] Additionally, or alternatively, the ultrasonic diagnostic system 100 of the example of FIG. 1 may include a magnetic positioning system or an attitude orientation reference system to detect the position of the human subject 102, the position of the ultrasonic transducer probe 104, or both. For example, the position and / or orientation of the ultrasonic transducer probe 104 can be determined by various micro electro-mechanical devices (MEMS) such as an accelerometer.

[0032] Additionally, or alternatively, the ultrasonic diagnostic system 100 may include an infrared scanning system configured to scan an illuminated object such as the human subject 102 in three dimensions. The infrared scanning system may include an infrared projector, a camera or a CMOS image sensor for detecting infrared rays interacting with the illuminated object, and a microchip including computer-executable instructions for spatially processing the object to be scanned. A suitable infrared scanning system includes the Light Coding (trademark) system included in the Kinect (trademark) gaming system. The infrared scanning system can complement the above-described optical tracking device and the above-described optical target, or can replace them in some applications.

[0033] In practice, an operator (not shown), such as an ultrasound technician, a physician, or another individual, operates the ultrasound transducer probe 104 in a manner that radiates sound waves into the body of the human subject 102. In the context of obtaining echo return data, which may alternatively be referred to as sonogram information or sonogram data herein, the operator initiates the scanning process by performing a first sonogram scan 136 at a selected location on the human subject 102. To conceptually illustrate the use of one example of the ultrasound diagnostic system 100, an examination of the spine of the human subject 102 is described. The sonogram scan is initiated at a first location, such as near the head of the human subject 102, which is a location lateral to the centerline of the spine of the human subject 102. Next, the operator moves the ultrasound transducer probe 104 in a substantially straight line across the spine of the human subject 102.

[0034] During the first sonogram scan 136, in one exemplary embodiment, sonogram information corresponding to a plurality of continuously acquired sonogram images is communicated from the ultrasound transducer probe 104 to the ultrasound interface 110. A time stamp corresponding to the time at which a sonogram image is acquired can be added to the individual sonogram images by the ultrasound transducer probe 104 to generate a time-indexed sonogram image information portion. Alternatively, the clock 126 can add time information to the acquired sonogram images to generate a time-indexed sonogram information portion.

[0035] Alternatively, the ultrasonic transducer probe 104 can provide a continuous stream of sonogram information (echo return data) corresponding to the return echoes detected by the ultrasonic transducer probe 104 during each scan. When a stream of data corresponding to the detected echoes is provided, time stamps are periodically added to and / or associated with specific portions of the streaming echo return data. Thus, the echo return data obtained at the start of the scan has an associated first time stamp corresponding to the data acquisition time, and later portions of the obtained echo return data are associated with later time stamps to reflect the time at which that echo return data was obtained by the ultrasonic transducer probe 104.

[0036] The associated time stamps identify the acquisition time of the sonogram image and / or the acquisition time of a portion of the sonogram echo data stream. The time-indexed sonogram image and / or the time-indexed sonogram echo data portion are, interchangeably herein, referred to as the time-indexed sonogram image information portion.

[0037] The time stamps associated with the time-indexed sonogram information can be expressed in real time or by using a reference time. The time stamp information is provided by an internal clock within the ultrasonic transducer probe 104. Alternatively, the clock 126 can add time stamp information to the obtained sonogram information when the time-indexed sonogram information is communicating from the ultrasonic interface 110 to the ultrasonic image data processor 112. The sonogram image time stamp has the same time reference as the corresponding time stamp associated with the time-indexed camera image simultaneously captured by the image capture device 114.

[0038] As is known in the art, the ultrasonic image data processor 112 processes the received sonogram information obtained during the first sonogram scan 136 into a time-indexed sonogram image information portion that can be used to render the first sonogram image. The time-indexed sonogram image information portion is communicated from the ultrasonic image data processor 112 to the image registration module 118.

[0039] The time stamp of each of the plurality of portions of the first sonogram image is correlated by the image registration module 118 with the corresponding time-indexed camera image. For each of the time stamps of one of the time-indexed sonogram image information portions, the corresponding camera image having the same or substantially the same time stamp is correlated with that particular time-indexed sonogram image information portion. Thus, between each portion of the first sonogram scan 136, the position and orientation of the ultrasonic transducer probe 104 with respect to the target 130 on the human subject 102 are determined. That is, the position and orientation of the ultrasonic transducer probe 104, and thus the position of each time-indexed sonogram image information portion on the body of the human subject 102, are determined by the image registration module 118. The position on the body of the human subject 102 is based on the position information determined from the corresponding time-indexed camera image having the same time stamp information. Thus, the position information for each associated position-indexed sonogram image information portion identifies the position of that first sonogram image portion on the body of the human subject 102.

[0040] As those skilled in the art will understand, the first sonogram image generated from the first sonogram scan 136 typically has a relatively narrow range (width) that includes only a width of 2.54 centimeters (1 inch) or more. Thus, after the first sonogram scan 136 is completed, the operator moves the position of the ultrasonic transducer probe 104 downward by a predefined increment distance (referred to herein as the "sonogram scan shift distance") for subsequent sonogram scans. Preferably, the sonogram scan shift distance is not greater than the sonogram image width of the sonogram image obtained during the first sonogram scan 136. Next, the operator performs a second sonogram scan 138 across the human subject 102.

[0041] The second sonogram scan 138 moves parallel to, or substantially parallel to, the first sonogram scan 136. Those skilled in the art will understand that some overlap can occur between the sonogram image information obtained during the second sonogram scan 138 and the first sonogram image information obtained during the first sonogram scan 136. In some situations, such overlap in sonogram image information is desirable. During subsequent construction of the 3D / 2D image and / or 3D / 2D model data, information determined from any overlapping portions of the sonogram images is merely duplicate and can be discarded, erased, or not used, and thus does not adversely affect the 3D / 2D image and / or 3D / 2D model data. In some embodiments, the overlapping information is combined to generate enhanced sonogram image information.

[0042] Next, the image registration module 118 generates a plurality of indexed sonogram image information portions based on the information received from the ultrasonic image data processor 112 and the optical tracking unit 116. For each processed sonogram scan, the indexed sonogram image information portion includes sonogram image information for a particular portion of the sonogram scan, optionally indexed information identifying the particular acquisition time of the sonogram image portion to which each time index is associated, and position information for each associated sonogram image portion identifying the position of the image portion on the body of the human subject 102.

[0043] Similarly, a third sonogram scan 142 adjacent to the second sonogram scan 138 can be acquired. The process of performing a continuous series of sonogram scans 140 is continued, with each successive sonogram scan separated from the previous sonogram scan by a predefined sonogram scan shift distance. The process of performing a series of parallel sonogram scans is continued across the portion of the human subject 102 being examined. In an exemplary example of examining the spine of the human subject 102, the sonogram scan process ends with a final sonogram scan 144 corresponding to the scan of the lower end of the spine of the human subject 102.

[0044] The above-described scanning process was described as a series of parallel sonogram scans 136-144 oriented perpendicular to the orientation of the spine of human subject 102. This scanning sequence is convenient because the operator of the ultrasonic transducer probe 104 can intuitively track the sonogram scans performed during the examination of human subject 102. One skilled in the art will understand that any sonogram scanning process can be used during the examination of human subject 102 because the orientation and position of the ultrasonic transducer probe 104 relative to the scanning position on human subject 102 can be easily determined. For example, the sonogram scan can be aligned diagonally with respect to the examination site of human subject 102. Cross-intersecting sonogram scans may be used for the examination. Even an elliptical or circular scanning motion can be used during the examination. All such various sonogram patterns will result in a complete 3D image and / or 3D data of the examination site of human subject 102 upon completion of the examination process.

[0045] The image processing algorithm module 120 receives from the image registration module 118 the plurality of position-indexed sonogram image information portions generated for each sonogram scan. The position-indexed sonogram image information portions received for each sonogram scan are stored in a suitable storage medium (not shown) or database 124. In an exemplary embodiment, each subsequently received position-indexed sonogram image information portion is stored during the scanning process.

[0046] At the end of the scanning process when the final position-indexed sonogram image information portion generated from the final sonogram scan 144 is received, the position-indexed sonogram image information for each individual sonogram scan is retrieved by the image processing algorithm module 120 for processing. The processing includes a plurality of processing steps.

[0047] The initial processing step performed by the image processing algorithm - module 120 is to aggregate or combine a plurality of individually - located - index - tagged sonogram image information portions into composite sonogram image information. When sonogram information and / or data are provided in separate image files, individual image frames are selected for processing. When sonogram information is provided as a continuous stream of data, the streaming echo - return data is analyzed using an appropriate sampler algorithm into the sonogram image portion. For example, one slice or frame is taken every 0.1 seconds from the streaming echo - return data and then saved for further processing. Any appropriate sampling time may be used. Any appropriate sampling application, currently known or later developed, that converts a continuous - time signal to a discrete - time signal can be used by an embodiment of the ultrasonic diagnostic system 100.

[0048] Each of the sonogram image information portions with individual position indexes is referenced to a reference position on the body of the human subject 102 (the reference position is determined from the position of the marker 130). Therefore, each portion of the sonogram image information portion with a position index can be ordered by a specific position on the body of the human subject 102 and then combined (or grouped) with adjacent portions of the previously acquired sonogram image information portion with a position index and the subsequently acquired sonogram image information portion with a position index. For example, the second sonogram image information portion with a position index generated from the second sonogram scan 138 is combined with the adjacent first position sonogram image information portion generated previously (from the first sonogram scan 136). Also, the second sonogram image information portion with a position index generated from the second sonogram scan 138 is combined with the adjacent third position sonogram image information portion generated previously (from the third sonogram scan 140). This combination of the sonogram image information portions with position indexes continues until all of the generated sonogram image information portions with position indexes are combined into a single composite sonogram image information file or data. This is called stitching of images in the art, and any suitable methodology for combining together the currently known or later developed sonogram image information portions with position indexes can be used by an embodiment of the ultrasonic diagnostic system 100.

[0049] In an alternative embodiment, when the sonogram image information portion with position index is received from the image registration module 118, composite sonogram image information is generated by combining each received sonogram image information portion with position index into the composite sonogram image information. In this specification, the composite sonogram image information is generated in real time or substantially in real time. As described in this specification, the graphic presentation of the composite sonogram image information can be presented to the operator when each of the continuous individual sonogram scans is performed. The "size" of the graphic presentation of the composite sonogram image information displayed increases as each successive sonogram scan is performed. Such immediate real-time or substantially real-time feedback can be particularly desirable to assist the operator in being able to fully cover a portion of the body of the human subject 102 being examined. That is, if a portion of the body is missed during the scan or the image information is unclear or damaged, the operator can re-scan the portion of the body of the human subject 102 so that the subsequently acquired composite sonogram image information portion is integrated into the composite sonogram image information.

[0050] Unless the position of the marker 130 on the human subject 102 has changed, after the final sonogram scan 144 is performed, additional sonogram scans can be performed by the operator. The sonogram image information acquired subsequently can be correlated with the previously acquired sonogram scan information. Thus, if a missed portion is identified later and / or additional image data for a particular region on the body is desired where enhanced clarity and / or improved resolution is desired, the operator can re-scan that particular region of the body of the human subject 102. The subsequently acquired composite sonogram image information is then integrated into the previously generated composite sonogram image information. Since the next sonogram scan is indexed both in time and position, subsequent scans need not be performed in parallel with the original sonogram scan. One or more subsequent sonogram diagnostic scans can be performed along any direction of the subject.

[0051] When the composite sonogram image information is generated, the composite sonogram image information can be stored in the database 124. The database 124 can be located locally or remotely. Once stored, the composite sonogram image information can be retrieved later for processing.

[0052] Furthermore, the image processing algorithm module 120 can process the composite sonogram image information into composite sonogram graphical information that can be used to render 3D images and / or 2D images. The composite sonogram graphical information can optionally be stored in the database 124.

[0053] Alternatively, or in addition, the generated composite sonogram graphical information can be communicated to the 3D / 2D visualization module 122. The 3D / 2D visualization module 122 processes (renders) the received composite sonogram graphical information into image information that can be communicated to the 3D / 2D stereoscopic display 108 or another suitable display device for presentation to an operator or another individual. Any suitable image rendering process known currently or developed later can be used by the 3D / 2D visualization module 122 to generate a presentable composite sonogram image. Alternatively, or in addition, the composite sonogram graphical information can be communicated to a remote display system 146 configured to render and present the composite sonogram image.

[0054] One of ordinary skill in the art will understand that once the 3D composite sonogram graphical information is generated from the composite sonogram image information, any suitable 3D image presentation algorithm can be used to display the body part of the human subject 102 being examined. The graphical image, an example of the spine of the human subject 102 in this specification, can be rotated and / or oriented in any way for viewing by an operator such as a specialist or another individual. Any suitable 3D processing algorithm known currently or developed later can be used to present an image generated from the composite sonogram graphical information.

[0055] Other analysis algorithms can be integrated into the image processing algorithm module 120 and / or operate with the image processing algorithm module 120. For example, from the perspective of evaluating the degree of scoliosis of the spine of the human subject 102, the spine modeling and measurement algorithm can be used to perform automatic measurement and analysis of the spine of the human subject 102 being examined.

[0056] Preferably, although not necessarily, when the ultrasound diagnostic system 100 is used to examine the bone structure of the human subject 102, the image processing algorithm module 120 includes a filtering algorithm that excludes non-bone type sonogram echo information from the composite sonogram image information (or sonogram image information with multiple position indexes). In this specification, the background information in the detected acoustic echo information is suppressed so that only the echoes from the bone structure of the human subject 102 are retained for analysis.

[0057] Alternatively, or additionally, other filtering algorithms can be used to identify and separate other tissues or structures of interest within the human subject 102 being examined. For example, if an image of a particular soft tissue of the human subject 102 is of interest, the echo information generated by nearby bone can be suppressed by a filtering algorithm such that the composite sonogram image information is filtered to generate a 3D and / or 2D graphical image of the tissue of interest.

[0058] The use of the ultrasonic transducer probe 104 has not provided satisfactory results because the sonogram images are too noisy and it is simply impossible to make a high level of distinction of the particular organ and / or anatomical structure of interest. By a novel approach of applying a particular filter to the sonogram information before attempting to construct a 3D image and / or 3D data, an embodiment of the ultrasonic diagnostic system 100 can identify the particular organ of interest. In this specification, an application example describes an examination apparatus (examinant) of the spine of the original human subject 102. The applied filter is configured to exclude sonogram information that is not related to the spine bone of interest.

[0059] Preferably, although not essential, an artificial intelligence (AI) algorithm can be used by the image processing algorithm module 120 to enhance the quality and / or reliability of the generated composite sonogram image information. Convolutional neural networks (CNNs) can be used for image processing by an embodiment of the ultrasonic diagnostic system 100. The AI algorithm learns to further process the received filtered sonogram information to improve the highlighting of the organ or anatomical structure of the object being examined. For example, but not limited to, the AI system can learn to identify specific anatomical landmarks of interest. Any suitable AI system and / or neural network, currently known or later developed, can be used by various embodiments of the ultrasonic diagnostic system 100.

[0060] For example, if a specific anatomical landmark on the spine of the human subject 102 is of interest, information identifying such a landmark can be provided to the image processing algorithm module 120, particularly to the AI algorithm. Over time, as more similar landmarks are identified by the AI algorithm within another composite sonogram image information obtained from another human subject 102, or even later within the sonogram information obtained from the original human subject 102, the AI algorithm can learn to identify the anatomical landmarks of interest.

[0061] In a preferred embodiment, the AI algorithm can optionally be configured to calculate and / or obtain measurement information regarding a particular anatomical structure of interest. In the case of an inspection apparatus for the spinal column of the human subject 102, an exemplary embodiment evaluates the orientation and / or position of a particular spinal bone. That is, the degree of curvature, rotation and / or inclination between individual spinal bones (vertebrae) is automatically determined. Measurements of angular displacement and / or positional displacement of the spinal bones can be determined. Embodiments of the ultrasound diagnostic system 100 can be trained with AI to determine any anatomical measurement value of interest for any organ and / or anatomical structure of the subject being examined.

[0062] Furthermore, when a graphical presentation of the composite sonogram image information is performed on the 3D or 2D display 108, the AI algorithm can generate an annotated image presenting one or more graphical artifacts to modify the graphical image to emphasize and / or present information to an operator or other interested party viewing the presented graphical image. From the perspective of an inspection of the spinal column of the human subject 102, the generated graphical artifacts can be used to provide a visual display of the determined measurements of angular displacement and / or positional displacement of the spinal bones.

[0063] Figure 2 is a graphical image 200 of the spine of test human subject 102 examined using an embodiment of the ultrasonic diagnostic system 100. Image 200 presents a graphical image of a part of the spine of the examined human subject 102. The displayed region 202 notifies the operator or another interested party that the very bright regions of image portion 202 indicate ultrasonic diagnostic images of specific bones within the spine of the examined human subject 102. The displayed region 204, similarly, indicates that the very bright regions of image portion 204 indicate ultrasonic diagnostic images of different bones within the spine of the examined human subject 102. The displayed region 206 is also a very bright region, but due to the brighter region 208, region 206 does not indicate bone characteristics. Region 206 is probably the surface of a ligament or muscle connecting the bones corresponding to regions 202 and 206. The displayed region 210, similarly, indicates that the darker shaded regions of the image portion indicate ultrasonic diagnostic images corresponding to regions beneath the bone surface. The bone surface reflects most of the ultrasonic waves, creating a darker region beneath the spine of the examined human subject 102.

[0064] Figure 3 is a graphical image 300 of the spine of test human subject 102 after the AI algorithm of the image processing algorithm module 120 analyzes a specific spinal bone and adds one or more graphical artifacts overlaid on the image of the spine of human subject 102. The graphical artifacts provide information to the operator or another interested party regarding the associated spinal condition or situation. For example, a single presented image of the spine of human subject 102 can be measured to confirm the curvature of the spine in all directions and planes. Additionally, or alternatively, selected planes of 3D images and / or 3D data can be used to generate 2D images and / or 2D data along the plane of interest.

[0065] The absence of graphical artifacts for regions 306, 308, and 310 (corresponding to regions 206, 208, and 210 in Figure 2, respectively) indicates that these regions of the human subject 102 being examined do not have the information of particular interest and / or are associated with other tissues that were not the subject of the examination. Therefore, no graphical artifacts were generated in these regions 306, 208, or 310.

[0066] Figure 4 is a conceptual diagram 400 of the human subject 102 showing the spine 402 and pelvic bone 404. Figure 5 is a conceptual diagram 500 of a 3D or 2D image generated from composite sonogram graphical information presenting images of the spine 402 and pelvic bone 404 of the human subject 102. The image 500 shows a first graphical artifact 502 (corresponding to the graphical artifact 302 in Figure 3) and a second graphical artifact 504 (corresponding to the graphical artifact 304 in Figure 3). There, the AI algorithm of the image processing algorithm module 120 presents diagnostic information useful to the viewing operator or another interested party.

[0067] In some embodiments, graphical artifacts can be generated to highlight specific anatomical features of interest in order to assist in the evaluation of the condition of the human subject 102 being examined. For example, a plurality of graphical artifacts 506 that highlight the protruding portions (bumps) on the outside of each spinal vertebra can be generated and overlaid on the generated spinal image. Various colors, shades, and / or illuminations can be used in the presented graphical artifacts 506 to assist the examiner in evaluating the condition of the spine of the human subject 102. In this simplified conceptual example, the graphical artifacts 506 are generated and presented only for four spinal vertebrae. However, the graphical artifacts 506 may be generated and presented for all of the spinal vertebrae, or for selected spinal vertebrae of interest. Further, using the interactive display 108 and an appropriate graphical user interface (GUI), the examiner can interactively select and / or manipulate any of the presented graphical artifacts 502, 504, and / or 506.

[0068] 3D or 2D images of the spine are generated by the image processing algorithm module 120 during further processing of the composite sonogram image information. In this specification, after extensive filtering, image information or data that identifies specific bones of the spine of the human subject 102 is identified with high resolution. Further, an AI algorithm may be learned to identify specific bones of the spine of the human subject 102. The AI accesses the appropriate skin (image data that graphically shows a more realistic image of the bone) for each specific bone of the spine of the human subject 102 and uses the accessed skin to create a more realistic graphical representation of the spine.

[0069] In some embodiments, the corresponding ideal structural image may be accessed from a database and overlaid on top of, or presented adjacent to, the image generated based on the sonogram examination. For example, image data of an ideal spine can be accessed. The image data of the ideal spine can then be scaled to correspond to the image of the spine of the human subject 102. The overlaid or adjacent images of the ideal spine can then be visually compared to the image of the spine of the examined human subject 102. The comparison can be performed using 3D or 2D images.

[0070] FIG. 6 is a block diagram of a programmable computing device suitable for use as part of an image processing system 600 implemented in an ultrasonic diagnostic system 100. The following paragraphs describe one suitable example of an image processing system, but those skilled in the art will understand that many different examples are possible. For example, the image processing system 600 can include an embedded software system, a stand-alone personal computer, and / or a networked computer system.

[0071] From the disclosure of the ultrasonic diagnostic system 100, those skilled in the art will recognize that various examples of the image processing system 600 can be implemented using electronic circuits configured to perform one or more functions. For example, using some embodiments of the present invention, the image processing system can be implemented using one or more application-specific integrated circuits (ASICs). However, in some examples, the components of various examples of the present invention can be implemented using a programmable computing device that executes firmware or software instructions, or by some combination of application-specific electronic circuits and firmware or software instructions executed on a programmable computing device.

[0072] Accordingly, FIG. 6 shows an exemplary embodiment of an image processing system 600, which is a computer that can be used to implement various embodiments of the present invention. As can be seen in this figure, the exemplary image processing system 600 has a computing unit 602. The computing unit 602 typically includes a processing unit 604 and a system memory 606. The processing unit 604 may be any type of processing device for executing software instructions, but conventionally would be a microprocessor device. The system memory 606 may include a read-only memory (ROM) 608 and a random access memory (RAM) 610. As will be understood by those skilled in the art, both the read-only memory (ROM) 608 and the random access memory (RAM) 610 can store software instructions executed by the processing unit 604.

[0073] The processing unit 604 and the system memory 606 are directly or indirectly connected to one or more peripheral devices via a bus 612 or an alternative communication structure. For example, the processing unit 604 or the system memory 606 may be directly or indirectly connected to additional memory storage devices such as a hard disk drive 614, a removable optical disk drive 616, a removable magnetic disk drive 618, and a flash memory card 620. The processing unit 604 and the system memory 606 may also be directly or indirectly connected to one or more input devices 622 and one or more output devices 624. The input device 622 may include, for example, a keyboard, a touch screen, a remote control pad, a pointing device (such as a mouse, a touch pad, a stylus, a trackball, or a joystick), a scanner, a camera, or a microphone. The output device 624 may include, for example, a monitor display, an integrated display, a television, a printer, a stereo, or a speaker.

[0074] Furthermore, computing unit 602 will be directly or indirectly connected to one or more network interfaces 626 for communicating with the network. This type of network interface 626 may also be referred to as a network adapter or a network interface card (NIC), but it converts data and control signals from computing unit 602 into network messages according to one or more communication protocols such as the Transmission Control Protocol (TCP), the Internet Protocol (IP), and the User Datagram Protocol (UDP). Since these protocols are well-known in the art, they will not be described in more detail herein. Interface 626 can use any suitable connection medium for connecting to a network, including, for example, a wireless transceiver, a power line adapter, a modem, or an Ethernet® connection.

[0075] In addition to the peripheral devices such as the input device, output device, and storage device specifically listed above, it should be understood that the computing device can be connected to various other peripheral devices that can perform input, output, and storage functions, or some combinations thereof. For example, computer 101 is often connected to a 3D ultrasound processor and a transducer system. In addition to the 3D ultrasound unit, computer 101 can be connected to or include one or more other peripheral devices such as a telephone, a fax machine, a router, etc.

[0076] For example, the telephone may be a wireless "smartphone" such as one that runs on the Android or iOS operating system. As is known in the art, this type of telephone communicates via a wireless network using radio frequency transmission. In addition to simple communication functions, a "smartphone" can provide a user with one or more data management functions such as sending, receiving, and viewing electronic messages (such as email messages, SMS text messages, etc.), recording or playing audio files, recording or playing image files (such as still or video files), and viewing or editing files using text (such as Microsoft Word or Excel files, or Adobe Acrobat files). Thanks to the data management capabilities of this type of telephone, a user can connect the telephone to the computing unit 602 to synchronize the maintained data.

[0077] Of course, as is well known in the art, additional peripheral devices may be included in or otherwise connected to a computing unit 602 of the type shown in FIG. 2. In some cases, the peripheral devices can be connected to the computing unit 602 permanently or semi-permanently. For example, in many computers, the computing unit 602, hard disk drive 614, removable optical disk drive 616, and display are semi-permanently housed in a single housing.

[0078] However, other peripheral devices may also be removably connected to the computing unit 602. The computing unit 602 can include, for example, one or more communication ports through which the peripheral devices can be connected to the computing unit 602 (either directly or indirectly via the bus 612). Thus, these communication ports may include parallel bus ports or serial bus ports such as the Universal Serial Bus (USB) standard, or the IEEE 1394 high-speed serial bus standard (e.g., Firewire port). Alternatively, or in addition, the computer 101 may include wireless data “ports” such as a Bluetooth® interface, a Wi-Fi interface, an infrared data port, and the like.

[0079] It should be understood that the computing devices used in accordance with various examples of the present invention may include more components other than, or in addition to, the computing unit 602 shown in FIG. 6. Further, fewer components than the computing unit 602, or a different combination of components than the computing unit 602, may be used in alternative embodiments. For example, some embodiments of the present invention can use one or more computing devices intended to have very specific functions such as a server computer. Thus, these computing devices can omit unnecessary peripheral devices such as a network interface 626, a removable optical disk drive 616, a printer, a scanner, an external hard drive, and the like. Some embodiments of the present invention can alternatively or additionally use computing devices intended to enable a wide variety of functions such as a desktop or laptop personal computer. These computing devices can have a desired combination of peripheral devices or additional components.

[0080] Furthermore, in another implementation example, the ultrasonic diagnostic system 100 can be used to subsequently call up previously generated images and / or data of the spine of the human subject 102. As described herein, the previous images and / or data can be stored in the ultrasonic diagnostic system 100 and / or a computer system of a local or remote database. Next, the ultrasonic diagnostic system 100 matches a current 3D image of the spine of the human subject 102 with the previously acquired images. In this exemplary application, the images to be compared can be used to assist an operator in evaluating the health condition and / or the effectiveness of treatment of the spine with respect to the human subject 102.

[0081] Examples of the ultrasonic diagnostic system 100 may be suitable for obtaining sonogram information from other animals, such as pets, livestock, zoo animals, etc., in addition to obtaining data or image information and generated 3D or 2D model information for scanned tissues, bones, or organs of the human subject 102. Examples of the ultrasonic diagnostic system 100 can also be used to obtain sonogram information from plants or other inanimate objects. For example, ancient artifacts or relics suitable for scanning using an ultrasonic transducer probe could be scanned using an example of the ultrasonic diagnostic system 100. Furthermore, sonographic scanning is often used to scan a fetus in the mother's womb. Examples of the ultrasonic diagnostic system 100 can also be used to scan these fetuses.

[0082] One skilled in the art can understand that an embodiment of the ultrasonic diagnostic system 100 may be configured to detect, distinguish, identify, and present other non-biological objects within the human subject 102. For example, pins, screws, fasteners, etc. made of metal or polymer may have been embedded in the human subject 102 during previous surgical procedures. Such objects can be identified and added to the generated 3D or 2D model information or data. Further, since the 3D or 2D model information or data can be generated in real time or almost in real time, the surgical instruments currently being used during the procedure being undergone by the human subject 102 can be identified. In this specification, the ultrasonic diagnostic system 100 can be used to simultaneously detect such surgical instruments together with the target organ or tissue.

[0083] One skilled in the art can understand that the ultrasonic diagnostic processor system 106 needs to be local or in proximity to the ultrasonic transducer probe 104 during the examination of the human subject 102. In this specification, the image capture device 114 needs to be local to the human subject 102 during the sonogram scanning process so as to capture images of the targets 130, 132 during the examination of the human subject 102. Such embodiments may be configured to be remotely located from the ultrasonic transducer probes 104 and one or more image capture devices 114. The ultrasonic diagnostic processor system 106 receives ultrasonic information and camera images via a suitable communication system that communicatively couples the ultrasonic diagnostic processor system 106 with the ultrasonic transducer probes 104 and one or more image capture devices 114. Further, such embodiments can be configured to receive information from a plurality of ultrasonic transducer probes 104 and one or more image capture devices 114, such that a plurality of human subjects 102 can be examined remotely and / or simultaneously. Further, the generated 3D / 2D images and / or 3D / 2D model data can be communicated back to the examination site for display on a display device disposed at the examination site.

[0084] Figure 7 is a flowchart showing a process used by an exemplary embodiment of an ultrasonic diagnostic system 100. The flow of camera image and sonogram image information is acquired (602) during sonogram scanning. A time stamp is added (604) to the camera image to generate a time-indexed camera image. The same or substantially the same time stamp is added to the corresponding portion of the sonogram image information to generate a time-indexed sonogram image information portion.

[0085] Each time-indexed camera image is processed (606) to determine corresponding time-indexed position information identifying a particular portion of the human subject 102 that was scanned during the time of the time stamp.

[0086] At the same time, the flow of sonogram image information is acquired (608) during a plurality of sonogram scans. A time stamp is added (610) to a portion of the sonogram image information to generate a time-indexed sonogram image information portion.

[0087] Next, for a selected time (612), the time-indexed position information for the selected time is combined with the time-indexed sonogram image information portion (acquired simultaneously with the time stamp) to generate a position-indexed sonogram image information portion for that particular portion of the sonogram information.

[0088] Next, each of the plurality of position-indexed sonogram image information portions is combined (616) to generate composite sonogram image information. The composite sonogram image information is then used (618) to generate 3D or 2D composite sonogram graphical information, which can be used (620) to render a 3D or 2D image on a display.

Industrial Applicability

[0089] The invention described in this application can be manufactured by various industrial processes including various mechanical, electrical, and pneumatic assembly techniques. Further, the invention described herein can be used in an industrial context including medical diagnostic applications.

[0090] The above-described invention can alternatively be explained according to the following non-limiting examples.

[0091] In a first embodiment for an ultrasonic diagnostic system, the system comprises an ultrasonic transducer probe configured to acquire sonogram image information from a plurality of successive ultrasonic scans of a human subject, the ultrasonic transducer probe generating sonogram image information between each of the plurality of ultrasonic scans; an ultrasonic image data processor communicatively coupled to the ultrasonic transducer probe, the ultrasonic transducer probe receiving sonogram image information from the ultrasonic transducer probe, the ultrasonic image data processor generating a plurality of time-indexed sonogram image information portions each including sonogram image information and a first time corresponding to the acquisition time of the sonogram image information; at least one image capture device capturing a time series of a series of camera images including both the human subject and the ultrasonic transducer probe, at least one first optical target disposed on the human subject being visible in each of the series of captured camera images, and at least one second optical target disposed on the ultrasonic transducer probe being visible in each of the series of captured camera images; an optical tracking unit communicatively coupled to the at least one image capture device, the optical tracking unit receiving each of the series of camera images from the at least one image capture device, each of the series of camera images including a time index indicating the acquisition time of the camera image by the at least one image capture device, the optical tracking unit determining the position of at least one first target on the human subject and the position of at least one second target on the ultrasonic transducer probe, the optical tracking unit determining time-indexed position information for each camera image based on the determined positions of the first optical target and the second optical target, the time-indexed position information identifying a portion of the human subject being scanned by the ultrasonic transducer probe based on the determined positions of at least one optical target on the human and at least one second optical target on the ultrasonic transducer probe, and the time-indexed position information including a second time; the optical tracking unit;An image registration module communicably connected to an ultrasonic image data processor and at least one image capture device, which selects position-indexed information having a second time that is the same as a first time for each time-indexed sonogram image information portion, and generates a plurality of position-indexed sonogram information portions based on each of the time-indexed sonogram image information portion and the corresponding time-indexed position information; and an image processing algorithm module communicably connected to the image registration module, which receives the position-indexed sonogram image information portion from the image registration module and combines the received plurality of position-indexed sonogram image information portions to generate composite sonogram image information.

[0092] In some examples of the first ultrasonic diagnostic system, the image processing algorithm module generates three-dimensional (3D) composite sonogram graphical information based on a plurality of composite sonogram image information. In such an example of the first ultrasonic diagnostic system, the system can include a 3D visualization module that generates an image based on the 3D composite sonogram graphical information, and the image can be presented on a display.

[0093] Some examples of the first ultrasonic diagnostic system include a clock that provides the acquisition time of ultrasonic information by an ultrasonic transducer probe, and the clock is within the ultrasonic image data processor. Certain examples include a clock that provides the acquisition time of a camera image captured by at least one image capture device, and the clock is within the ultrasonic image data processor.

[0094] In some examples, the ultrasonic transducer probe and at least one image capture device are disposed at a first location where a human subject is being examined, and at least the optical tracking unit, image registration module, and image processing algorithm module are at a second location remote from the first location, and the optical tracking unit, image registration module, and image processing algorithm module are operable to receive information from a plurality of other ultrasonic transducer probes and associated at least one image capture device at different first locations where different human subjects are being examined.

[0095] In a first embodiment of a method for examining a human subject using an ultrasonic diagnostic system, the method includes receiving a plurality of successive ultrasonic scans of the human subject from an ultrasonic transducer probe; generating a plurality of time-indexed sonogram image information portions each including sonogram image information from a portion of the ultrasonic scan and a first time corresponding to the acquisition time of the sonogram image information portion; capturing a series of camera images in time series including both the human subject and the ultrasonic transducer probe using at least one image capturing device, each of the series of camera images including a second time indicating the acquisition time of the camera image by the at least one image capturing device; determining time-indexed position information for each camera image, the time-indexed position information identifying the position of a portion of the human subject scanned by the ultrasonic transducer probe and including the associated second time; selecting, for each time-indexed sonogram image information portion, time-indexed position information having the same second time as the first time of the sonogram image information portion; generating a plurality of position-indexed sonogram information portions based on each time-indexed sonogram image information portion and the corresponding time-indexed position information; and generating composite sonogram image information by combining the received plurality of position-indexed sonogram image information portions.

[0096] In some examples of embodiments of the first method, at least one first optical target is disposed on a human subject and is visible in each of a series of captured camera images, and at least one second optical target disposed on an ultrasonic transducer probe is visible in each of a series of captured camera images, determining time-indexed position information for each camera image includes determining the position of at least one first target on the human subject and the position of at least one second target on the ultrasonic transducer probe, and the position of a portion of the human subject being scanned by the ultrasonic transducer probe is determined based on the determined position of at least one first target and the determined position of at least one second target.

[0097] In a particular example of an embodiment of the first method, generating a plurality of time-indexed sonogram image information portions includes adding a first time to each of the time-indexed sonogram image information portions using a clock of an ultrasonic image data processor that receives a plurality of successive ultrasonic scans from an ultrasonic transducer probe disposed remotely. Additionally, or alternatively, after capturing a series of camera images in time series, the method includes adding a second time to each of the camera images using a clock of an ultrasonic image data processor that receives a plurality of successive ultrasonic scans from an ultrasonic transducer probe disposed remotely.

[0098] Some examples of embodiments of the first method further include generating three-dimensional (3D) compound sonogram graphical information based on the generated compound sonogram image information. These examples further include generating a 3D image based on the 3D compound sonogram graphical information and presenting the 3D image on a display. Further, in such examples, after all of a plurality of successive ultrasound scans of a human subject have been received from an ultrasound transducer probe, a 3D image can be generated and presented on the display. Additionally, or alternatively, a 3D image can be generated and presented on the display as each of a plurality of successive ultrasound scans of a human subject is received from the ultrasound transducer probe.

[0099] Examples of embodiments of the first method further include generating a 2D image based on the 3D compound sonogram graphical information and presenting the 2D image on a display.

[0100] It should be emphasized that the above-described examples of the ultrasound diagnostic system 100 are merely possible examples of embodiments of the present invention. Many modifications and variations can be made to the above-described examples. All such modifications and variations are intended to be included herein within the scope of the present disclosure and protected by the following claims.

[0101] Furthermore, the above disclosure includes a plurality of individual inventions having independent utility. Each of these inventions is disclosed in a specific form, but the specific embodiments disclosed and exemplified above should not be considered in a limiting sense since numerous variations are possible. The present invention includes all novel and non-obvious combinations and sub-combinations of various elements, features, functions, and / or properties that are inherent to those skilled in the art related to the invention disclosed above and such inventions. Where the present disclosure or the claims filed therewith describe an element as "one," "a first," or any such equivalent term, the present disclosure or claims should be understood to incorporate one or more such elements and not to require or exclude two or more such elements.

[0102] The applicant reserves the right to file claims directed to combinations and sub-combinations of the disclosed inventions that are believed to be novel and non-obvious. Inventions embodied in other combinations and sub-combinations of features, functions, elements, and / or properties may be claimed through amendment of those claims or the presentation of new claims in the present application or related applications. Such amended or new claims shall be considered to be within the scope of the invention described herein whether they are directed to the same invention or a different invention and whether they are broader, narrower, or equal in scope to the original claims.

Claims

[Claim 1] an ultrasound transducer probe configured to acquire sonogram image information from a plurality of successive ultrasound scans of a human subject, the ultrasound transducer probe generating sonogram image information during each of the plurality of ultrasound scans; an ultrasound image data processor communicatively coupled to the ultrasound transducer probe, the ultrasound transducer probe receives the sonogram image information from the ultrasound transducer probe; the ultrasound image data processor generating a plurality of time-indexed sonogram image information portions, each portion including the sonogram image information and a first time corresponding to a time of acquisition of the sonogram image information; at least one image capture device for capturing a time-sequential series of camera images including both the human subject and the ultrasound transducer probe; at least one first optical target disposed on the human subject is visible in each one of the series of captured camera images; at least one image capture device, wherein at least one second optical target disposed on the ultrasound transducer probe is visible in each one of the series of captured camera images; an optical tracking unit communicatively coupled to the at least one image capture device, receiving each of the sequence of camera images from the at least one image capture device; each of the sequence of camera images includes a time index indicating a time of acquisition of the camera image by the at least one image capture device; the optical tracking unit determines a position of the at least one first target on the human subject and a position of the at least one second target on the ultrasound transducer probe; the optical tracking unit determines time-indexed position information for each camera image based on the determined positions of the first optical target and the second optical target; the time-indexed position information identifies a portion of the human subject being scanned by the ultrasound transducer probe based on the determined positions of the at least one first optical target on the human subject and the at least one second optical target on the ultrasound transducer probe; an optical tracking unit, the time-indexed location information including a second time; an image registration module communicatively coupled to the ultrasound image data processor and the at least one image capture device, selecting, for each time-indexed sonogram image information portion, location-indexed information having the second time that is the same as the first time; an image registration module that generates a plurality of location-indexed sonogram image information portions based on the time-indexed sonogram image information portions and each of the corresponding time-indexed location information; an image processing algorithm module communicatively coupled to the image registration module, receiving the positionally indexed sonogram image information portion from the image registration module; combining the received plurality of positionally indexed sonogram image information portions to generate composite sonogram image information; an image processing algorithm module that generates three-dimensional (3D) composite sonogram graphical information based on the plurality of composite sonogram image information; a 3D visualization module that generates an image based on the 3D composite sonogram graphical information, the image being presentable on a display; and An ultrasound diagnostic system comprising:

Images