Method and system for generating dynamic 3d ultrasound images

Abstract

The invention relates to a method and system for generating dynamic 3D ultrasound images. Various methods and systems for a medical imaging system are provided. In one embodiment, a method for generating a dynamic three-dimensional (3D) ultrasound image includes acquiring dynamic 3D ultrasound image data; generating a plurality of 3D rib gap segments from the dynamic 3D ultrasound data; generating a dynamic panoramic 3D ultrasound image depicting the plurality of dynamic 3D rib gap segments in an anatomical order; synchronizing the plurality of dynamic 3D rib gap segments in time; and outputting the dynamic panoramic 3D ultrasound image for display.

Description

Technical Field

[0001] Embodiments of the subject matter disclosed herein relate to ultrasonic imaging and, in particular, to visualizing the intercostal spaces of a subject as dynamic three-dimensional (3D) images. Background Art

[0002] Ultrasonic imaging systems typically include an ultrasonic probe applied to a patient's body and a workstation or device operatively coupled to the ultrasonic probe. During a scan, the ultrasonic probe can be controlled by an operator of the system and is configured to transmit and receive ultrasonic signals that are processed by the workstation or device into ultrasonic images. The workstation or device can display the ultrasonic images and a plurality of user-selectable inputs via a display device. An operator or other user can interact with the workstation or device to analyze the images displayed on and / or selected from the plurality of user-selectable inputs.

[0003] As an example, ultrasonic imaging can be used to examine a patient's lungs due to, for example, the ease of use of ultrasonic imaging systems at the point of care and the resource availability relative to chest x-rays or chest computed tomography (CT) scans. Additionally, ultrasonic imaging systems do not expose patients to radiation. Lung ultrasonic imaging, also known as lung ultrasonography, involves interpreting the topography of the lung pleura for diagnostic purposes. The ultrasonic image data captured by an ultrasonic imaging system can be static or dynamic image data and can be two-dimensional (2D) or three-dimensional (3D) image data. Summary of the Invention

[0004] This Summary of the Invention introduces concepts that are more fully described in the Detailed Description below. It should not be used to determine the essential features of the claimed subject matter nor to limit the scope of the claimed subject matter. In one aspect, a method for generating a dynamic three-dimensional ultrasonic image includes: acquiring dynamic three-dimensional ultrasonic image data; generating a plurality of three-dimensional intercostal space segments from the dynamic three-dimensional ultrasonic data; generating a dynamic three-dimensional ultrasonic image depicting the plurality of three-dimensional intercostal space segments in anatomical order; synchronizing the plurality of dynamic three-dimensional intercostal space segments in time; and outputting the dynamic three-dimensional ultrasonic image for display.

[0005] It should be understood that the above Summary is provided to introduce in a simplified form a selection of concepts that are further described in the Detailed Description below. This is not meant to identify the key features or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims that follow the Detailed Description. Additionally, the claimed subject matter is not limited to embodiments that solve any disadvantages noted above or in any part of this disclosure. Brief Description of the Drawings

[0006] The present disclosure will be better understood by reading the following description of non-limiting embodiments with reference to the accompanying drawings, in which:

[0007] Figure 1 shows a schematic block diagram of an ultrasonic imaging system according to an embodiment;

[0008] Figure 2 is a schematic diagram illustrating an image processing system for processing three-dimensional (3D) ultrasonic image data according to an embodiment;

[0009] Figure 3 shows an ultrasonic image probe of an ultrasonic imaging system (such as Figure 1 the ultrasonic imaging system);

[0010] Figure 4 illustrates the thoracic cavity of an imaging subject according to an embodiment;

[0011] Figure 5 shows a schematic flow chart of a method for generating a dynamic 3D ultrasonic image according to an embodiment;

[0012] Figure 6 shows a flow chart of a method for generating a dynamic 3D ultrasonic image according to an embodiment;

[0013] Figure 7 shows an example annotated 3D rib space segment that can be used to generate a 3D pleural surface according to an embodiment;

[0014] Figure 8 shows a first example dynamic panoramic 3D ultrasonic image according to an embodiment;

[0015] Figure 9 shows a second example dynamic 3D ultrasonic image according to an embodiment; and

[0016] Figure 10 shows an example display including a dynamic 3D ultrasonic image according to an embodiment. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will now be described by way of example with reference to Figures 1 to 10 to Figures 1 to 10Relates to various embodiments for generating dynamic three-dimensional ultrasound images from 3D ultrasound image data. The ultrasound probe of an ultrasound imaging system can be configured to capture static and / or dynamic images, where the dynamic images include multiple image frames acquired at different time points. Displaying the ultrasound image data as a dynamic image allows visualization of how the anatomical structure of the imaged subject changes over time. The size of the field of view (FOV) of the ultrasound probe determines the maximum area (e.g., in 2D ultrasound image data) and / or maximum volume (e.g., in 3D ultrasound image data) that can be acquired in a single frame of the ultrasound image data. Thus, due to the size of the FOV, a single frame of the ultrasound image data may not capture one or more items of interest, especially when imaging a large organ and / or large region of the body of the imaged subject. To minimize the likelihood of missing one or more items of interest in the ultrasound image data, especially when the imaged organ is relatively large compared to the FOV, some conventional ultrasound imaging systems are configured to generate panoramic views. For example, the size of the FOV of an ultrasound probe used to capture image data of a patient's chest can be set to capture one to two intercostal spaces at a time. The panoramic view can be generated from multiple ultrasound images acquired at different spatial positions. Due to the constraints imposed by the size of the FOV, the ultrasound images can be combined or stitched together to provide a single panoramic view that covers a larger area or volume than the area or volume covered in a single ultrasound image.

[0018] Additionally, an ultrasound imaging system can capture two-dimensional (2D) and / or three-dimensional (3D) ultrasound image data. For example, the ultrasound probe of an ultrasound imaging system can be configured to acquire dynamic 3D ultrasound image data. Some conventional methods for displaying 3D ultrasound images based on 2D ultrasound image data can be computationally intensive because the 2D ultrasound image data can be processed and modified to generate a 3D image from a stack of 2D ultrasound image data. As another example, a 3D mesh can be generated from the 2D ultrasound image data, and a 3D image can be generated from the 3D mesh. Thus, when a 3D ultrasound image is desired, it may be advantageous to perform an imaging scan using an ultrasound probe capable of capturing 3D ultrasound image data.

[0019] In an imaging scan for acquiring dynamic 3D ultrasound image data, there may be challenges in displaying items or regions of interest captured in the dynamic 3D ultrasound image data. For example, the dynamic 3D ultrasound image data can be acquired continuously (e.g., as an uninterrupted sequence of image frames), or can be acquired as multiple separate sequences of multiple image frames (e.g., where each sequence of the multiple sequences can have the same or different numbers of image frames). As described above, the size of the ultrasound probe FOV may limit the area or volume of the imaging subject captured in a single image. Thus, when moving the ultrasound probe across the outer surface of the imaging subject to capture image data of all regions of interest, multiple dynamic 3D ultrasound images can be captured. In conventional ultrasound imaging methods, each of the multiple dynamic 3D ultrasound images can be stored and displayed as a separate image, and the user can navigate and select between one or more images to be displayed and viewed for analysis and diagnosis. Additionally, the dynamic 3D ultrasound image data can show regions of interest of the imaging subject in an order different from the true anatomical order, and / or can display regions of interest for different durations. Thus, segments of the dynamic 3D ultrasound image data showing different regions of interest may vary between various imaging subject vital signs (such as the respiratory cycle or heart rate). It is known to temporally synchronize and temporally scale dynamic 2D images (e.g., videos) based on parameters such as the respiratory cycle or heart rate in a panoramic view such that the amount of playback time for each video in the panoramic view is the same. However, this method may not be sufficient to display dynamic 3D images in a panoramic view applicable to the imaging subject's anatomy.

[0020] A dynamic 3D ultrasound image can be generated from the dynamic 3D ultrasound image data captured by an imaging system (such as Figure 1 the ultrasound imaging system shown). Since the processes described herein can be applied to preprocessed imaging data and / or processed images, the term "image" is generally used throughout this disclosure to represent preprocessed and partially processed image data (e.g., pre-beamformed RF or I / Q data, pre-scan-converted RF data) as well as fully processed images (e.g., scan-converted and filtered images ready for display). Figure 2 An example image processing system that can be used to generate a dynamic 3D ultrasound image is shown in Figure 1 The ultrasound imaging system of Figure 3 includes an ultrasound probe, an example of which is shown in Figure 4 The imaging scan for capturing 3D ultrasound image data is performed by moving the ultrasound probe across the outer surface of an imaging subject (such as a patient). Methods for generating dynamic 3D ultrasound images of the intercostal spaces, internal lung surfaces, and other structures of a patient's chest are described herein, examples of which are shown in Figure 5A flowchart showing a schematic method for generating a dynamic 3D ultrasound image from 3D ultrasound image data is shown. Figure 6 A flowchart showing a more detailed method for generating a dynamic 3D ultrasound image from dynamic 3D ultrasound image data, the method comprising: generating a plurality of 3D rib interspace segments from the dynamic 3D ultrasound image data; identifying and exposing the pleural surface of each 3D rib interspace segment in the 3D rib interspace segments; and sorting the plurality of 3D rib interspace segments in anatomical order. The plurality of 3D rib interspace segments can be generated using rib shadowing and sorted in anatomical order using the pleural surface characteristics of each 3D rib interspace segment. Figure 7 An example annotated 3D rib interspace segment that can be used to generate a dynamic 3D ultrasound image is shown. Figure 8 A first example dynamic 3D ultrasound image shown in a panoramic view is shown. Figure 9 A second example dynamic 3D ultrasound image shown in a perspective view is shown. Figure 10 Shown for example Figure 1 An example display of an ultrasound imaging system such as Figure 9 A view of a second example dynamic 3D ultrasound image. In this way, 3D images of multiple rib interspace segments and other thoracic ultrasound images can be visualized in a single dynamic image.

[0021] Displaying data of interest in a single dynamic 3D ultrasound image (e.g., a 3D image showing multiple rib interspace segments with exposed pleural surfaces) allows a user to view the most relevant data without having to navigate, select, and analyze between multiple ultrasound images. The navigation speed of the user through various views (e.g., various rib interspace segments) can be increased because the method for guiding the display (e.g., navigation) described herein frees the user from selecting an image of a rib interspace segment from an array of all images of 3D ultrasound image data that can be displayed simultaneously or individually. In conventional methods, identifying a region of interest (e.g., a rib interspace of 3D ultrasound image data that potentially has an abnormality that may indicate a disease state) can include zooming in and / or out of one or more ultrasound images multiple times to compare the ultrasound images of a set of 3D ultrasound image data. Additionally, since the 3D ultrasound image data is dynamic (e.g., includes multiple frames), conventional display and analysis methods may require the user to further navigate between the durations of a single dynamic ultrasound image to view the details of the 3D ultrasound image. Further, the user can navigate between multiple dynamic ultrasound images (e.g., showing different regions of interest) and their durations to compare potential abnormalities between the multiple dynamic ultrasound images. This can pose further challenges because each of the multiple dynamic ultrasound images may have a different duration. Instead of using the methods described herein to sequentially select, zoom in, and analyze multiple 3D ultrasound images to identify relevant medical information, the user can view a display of a display device that has a single dynamic 3D ultrasound image displayed thereon, the single dynamic 3D ultrasound image including dynamic 3D ultrasound data captured from multiple regions of an imaging subject arranged in anatomical order (e.g., the multiple regions may not be capturable simultaneously according to the FOV of the ultrasound probe).

[0022] Reference Figure 1 , shows a schematic diagram of an ultrasound imaging system 100 according to an embodiment of the present disclosure. However, it can be understood that the embodiments described herein can be implemented using other types of medical imaging modalities (e.g., magnetic resonance imaging, computed tomography, positron emission tomography, etc.). The ultrasound imaging system 100 includes a transmit beamformer 101 and a transmitter 102 that drives transducer elements 104 within a transducer array (referred to herein as an ultrasound probe 106) to transmit pulsed ultrasound signals (referred to herein as transmit pulses) into an imaging subject's body (e.g., as relative to Figure 4As further described), according to an embodiment, the ultrasound probe 106 can be a 1.5-dimensional (1.5D) probe, a 3D probe, or any ultrasound probe capable of performing live 3D imaging, such as a matrix array probe. The transducer element 104 can be composed of a piezoelectric material. When a voltage is applied to the piezoelectric material, the piezoelectric material physically expands and contracts, thereby emitting ultrasonic spherical waves. In this way, the transducer element 104 can convert an electronic transmit signal into an acoustic transmit beam.

[0023] After the transducer element 104 of the ultrasound probe 106 emits a pulsed ultrasound signal into the (patient's) body, the pulsed ultrasound signal is backscattered from structures within the body (such as blood cells or muscle tissue) to generate echoes that return to the transducer element 104. These echoes are converted into electrical signals or ultrasound data by the transducer element 104, and the electrical signal is received by the receiver 108. The electrical signal representing the received echoes passes through a receive beamformer 110 that performs beamforming and outputs ultrasound data, which can be in the form of radio frequency (RF) signals. Additionally, the transducer element 104 can generate one or more ultrasound pulses based on the received echoes to form one or more transmit beams.

[0024] According to some embodiments, the ultrasound probe 106 can include electronic circuitry to perform all or part of transmit beamforming and / or receive beamforming. For example, all or part of the transmit beamformer 101, transmitter 102, receiver 108, and receive beamformer 110 can be located within the ultrasound probe 106. In the present disclosure, the term "scanning" can also be used to refer to the process of acquiring data by transmitting and receiving ultrasound signals. In the present disclosure, the term "data" can be used to refer to one or more data sets acquired using an ultrasound imaging system.

[0025] The user interface 115 can be used to control the operation of the ultrasound imaging system 100, including controlling the input of patient data (e.g., patient history), changing scan or display parameters, initiating a probe repolarization sequence, etc. The user interface 115 can include one or more of the following: a rotary element, a mouse, a keyboard, a trackball, hard keys linked to specific actions, soft keys that can be configured to control different functions, and a graphical user interface displayed on the display device 118. In some embodiments, the display device 118 can include a touch-sensitive display, and thus the display device 118 can be included in the user interface 115.

[0026] The ultrasonic imaging system 100 further includes a processor 116 that controls the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110. The processor 116 communicates electronically (e.g., communicatively coupled) with the ultrasonic probe 106. As used herein, the term "communicates electronically" can be defined to include both wired and wireless communication. The processor 116 can control the ultrasonic probe 106 to acquire data according to instructions stored in the memory of the processor and / or the memory 120. As an example, the processor 116 controls which transducer elements in the transducer element 104 are active and the shape of the beam emitted from the ultrasonic probe 106. The processor 116 also communicates electronically with the display device 118, and the processor 116 can process data (e.g., ultrasonic data) into an image for display on the display device 118. According to an embodiment, the processor 116 can include a central processing unit (CPU). According to other embodiments, the processor 116 can include other electronic components capable of performing processing functions, such as a digital signal processor, a field programmable gate array (FPGA), or a graphics board. According to other embodiments, the processor 116 can include a plurality of electronic components capable of performing processing functions. For example, the processor 116 can include two or more electronic components selected from a list of electronic components, including: a central processor, a digital signal processor, a field programmable gate array, and a graphics board. According to another embodiment, the processor 116 can further include a complex demodulator (not shown) that demodulates RF data and generates raw data. In another embodiment, the demodulation can be performed earlier in the processing chain.

[0027] The processor 116 is adapted to perform one or more processing operations according to a plurality of alternative ultrasonic modalities of the data. In one example, the data can be processed in real time during a scan session as the echo signals are received by the receiver 108 and sent to the processor 116. For the purposes of this disclosure, the term "real time" is defined to include procedures performed without any intentional delay (e.g., substantially as they occur). For example, embodiments can acquire images at a real-time rate of 7 frames per second to 20 frames per second. The ultrasonic imaging system 100 can acquire 3D ultrasonic image data of one or more planes at a significantly faster rate. However, it should be understood that the real-time frame rate can depend on the length of time (e.g., duration) spent acquiring and / or processing each frame of data for display. Therefore, when acquiring a relatively large amount of data, the real-time frame rate may be slower. Thus, some embodiments can have a real-time frame rate significantly faster than 20 frames per second, while other embodiments can have a real-time frame rate below 7 frames per second.

[0028] In some embodiments, data may be temporarily stored in a buffer (not shown) during a scan session and processed less in real time during live or offline (e.g., frozen) operations. Some embodiments of the present disclosure may include multiple processors (not shown) to handle processing tasks handled by processor 116 according to the exemplary embodiments described above. For example, before displaying an image, a first processor may be utilized to demodulate and extract the RF signal, while a second processor may be used to further process the data (by augmenting the data as further described herein). It should be understood that other embodiments may use different processor arrangements.

[0029] The ultrasound imaging system 100 may continuously acquire data, for example, at a frame rate of 10 Hz to 30 Hz (e.g., 10 frames per second to 30 frames per second). Images generated from the data may be refreshed on the display device 118 at a similar frame rate. Other embodiments may acquire and display data at different rates. For example, depending on the size of the frame and the intended application, some embodiments may acquire data at a frame rate less than 10 Hz or greater than 30 Hz. The memory 120 may store the processed frames of the acquired data. In an exemplary embodiment, the memory 120 has sufficient capacity to store at least several seconds' worth of ultrasound data frames. The data frames are stored in a manner that facilitates retrieval according to their acquisition order or time. The memory 120 may include any known data storage medium.

[0030] In various embodiments of the present disclosure, data may be processed by processor 116 in different mode-related modules to generate dynamic 3D ultrasound images. When multiple images are acquired, processor 116 may also be configured to stabilize or register the images. For example, one or more modules may generate B-mode, color Doppler, M-mode, color M-mode, color flow imaging, spectral Doppler, elastography, tissue velocity imaging (TVI), strain, and strain rate, etc., as well as combinations thereof. As an example, one or more modules may process color Doppler data, which may include conventional color flow Doppler, power Doppler, and high-definition (HD) flow Doppler, etc. Image lines and / or frames are stored in the memory and may include timing information indicating the time at which the image lines and / or frames are stored in the memory. These modules may include, for example, a scan conversion module that performs a scan conversion operation to convert the acquired images from beam space coordinates to display space coordinates. A video processor module may be provided that reads the acquired images from the memory and displays the images in real time while a procedure (e.g., ultrasound imaging) is being performed on the patient. The video processor module may include a separate image memory, and the ultrasound images may be written to the image memory for reading and display by the display device 118.

[0031] In addition, the components of the ultrasound imaging system 100 may be coupled to each other to form a single structure, may be separate but located in a common room, or may be remote from each other. For example, one or more of the modules described herein may operate in a data server that has a different remote location relative to other components of the ultrasound imaging system 100, such as the ultrasound probe 106 and the user interface 115. Optionally, the ultrasound imaging system 100 may be a single system that is capable of being moved (e.g., portably) from one room to another. For example, the ultrasound imaging system 100 may include wheels or may be transportable on a cart, or may include a handheld device.

[0032] For example, in various embodiments of the present disclosure, one or more components of the ultrasound imaging system 100 may be included in a portable handheld ultrasound imaging device. For example, the display device 118 and the user interface 115 may be integrated into the outer surface of the handheld ultrasound imaging device, which may also include a processor 116 and a memory 120 therein. The ultrasound probe 106 may include a handheld probe that communicates electronically with the handheld ultrasound imaging device to collect raw ultrasound data. The transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 may be included in the same or different portions of the ultrasound imaging system 100. For example, the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 may be included in the handheld ultrasound imaging device, the probe, and combinations thereof.

[0033] See Figure 2 , an example medical image processing system 200 is shown. In some embodiments, the medical image processing system 200 is incorporated into a medical imaging system such as an ultrasound imaging system (e.g., Figure 1 the ultrasound imaging system 100), an MRI system, a CT system, a single photon emission computed tomography (SPECT) system, etc. In some embodiments, at least a portion of the medical image processing system 200 is disposed at a device (e.g., an edge device or a server) that is communicatively coupled to the medical imaging system via a wired and / or wireless connection. In some embodiments, the medical image processing system 200 is disposed at a separate device (e.g., a workstation) that may receive images from the medical imaging system or from a storage device that stores images generated by the medical imaging system. The medical image processing system 200 may include an image processor 231, a user input device 232, and a display device 233. For example, the image processor 231 may be operably / communicatively coupled to the user input device 232 and the display device 233.

[0034] The image processor 231 includes a processor 204 that is configured to execute machine-readable instructions stored in a non-transitory memory 206. The processor 204 can be a single-core or multi-core processor, and the program executed by the processor 204 can be configured for parallel processing or distributed processing. In some embodiments, the processor 204 may optionally include separate components distributed across two or more devices, which may be located remotely and / or configured for cooperative processing. In some embodiments, one or more aspects of the processor 204 may be virtualized and executed by a remotely accessible networked computing device configured in a cloud computing configuration. In some embodiments, the processor 204 may include other electronic components capable of performing processing functions, such as a digital signal processor, a field-programmable gate array (FPGA), or a graphics board. In some embodiments, the processor 204 may include multiple electronic components capable of performing processing functions. For example, the processor 204 may include two or more electronic components selected from a plurality of possible electronic components including: a central processing unit, a digital signal processor, a field-programmable gate array, and a graphics board. In yet additional embodiments, the processor 204 may be configured as a graphics processing unit (GPU), including a parallel computing architecture and parallel processing capabilities.

[0035] In Figure 2 the illustrated embodiment, the non-transitory memory 206 stores a 3D generation module 212 and medical image data 214. The 3D generation module 212 includes one or more algorithms for processing input medical images from the medical image data 214. Specifically, as described with respect to Figure 5 and Figure 6 , the 3D generation module 212 can generate a dynamic 3D ultrasound image from dynamic 3D ultrasound image data. For example, the 3D generation module 212 may include one or more image recognition algorithms, shape or edge detection algorithms, gradient algorithms, etc. for processing 3D ultrasound image data. Additionally or alternatively, the 3D generation module 212 may store instructions for implementing a neural network, such as a convolutional neural network, to detect the pleural surface / pleura captured in the medical image data 214 in real time. For example, the 3D generation module 212 may include a trained and / or untrained neural network and may also include training routines or parameters (e.g., weights and biases) associated with one or more neural network models stored therein. In some embodiments, the 3D generation module 212 may evaluate the medical image data 214 during real-time acquisition of the medical image data. Additionally or alternatively, the 3D generation module 212 may evaluate the medical image data 214 offline rather than in real time.

[0036] For example, when the medical image data 214 includes lung ultrasound data, the identified anatomical features may include the lung pleura, which may be identified by the 3D generation module 212 via edge detection techniques and / or gradient changes based on pleural sliding. Additionally, the 3D generation module 212 may identify multiple intercostal spaces via edge detection techniques and / or gradient changes. As will be described in detail with respect to Figure 5 and Figure 6 , the detection of intercostal spaces may assist in generating multiple intercostal space segments, and the detection of pleural localization may assist in indicating areas of the ultrasound image that are noise and may be removed to expose the pleural surface.

[0037] Optionally, the image processor 231 may be communicatively coupled to a training module 210 that includes instructions for training one or more machine learning models stored in the 3D generation module 212. The training module 210 may include instructions that, when executed by a processor, cause the processor to build a model (e.g., a mathematical model) based on sample data to make predictions or decisions regarding the detection and classification of anatomical irregularities without explicit programming of conventional algorithms that do not utilize machine learning. In one example, the training module 210 includes instructions for receiving a training data set from the medical image data 214. The training data set includes a set of medical images, associated ground truth labels / images, and associated model outputs for training one or more machine learning models stored in the 3D generation module 212. The training module 210 may receive medical images, associated ground truth labels / images, and associated model outputs for training one or more machine learning models from sources other than the medical image data 214, such as other image processing systems, the cloud, etc. In some embodiments, one or more aspects of the training module 210 may include a remotely accessible networked storage device configured in a cloud computing configuration. Additionally, in some embodiments, the training module 210 is included in the non-transitory memory 206. Additionally or alternatively, in some embodiments, the training module 210 may be used to generate the 3D generation module 212 offline and away from the image processing system 200. In such embodiments, the training module 210 may not be included in the image processing system 200, but may generate data stored in the image processing system 200. For example, the 3D generation module 212 may be pre-trained at the manufacturing location using the training module 210.

[0038] The non-transitory memory 206 also stores medical image data 214. The medical image data 214 includes, for example, functional images and / or anatomical images captured by imaging modalities such as ultrasound imaging systems, MRI systems, CT systems, PET systems, etc. As an example, the medical image data 214 may include ultrasound images, such as lung ultrasound images. Additionally, the medical image data 214 may include one or more of 2D images, 3D images, static single-frame images, and multi-frame image loops (e.g., movies).

[0039] In some embodiments, the non-transitory memory 206 may include components disposed at two or more devices, which may be located remotely and / or configured for collaborative processing. In some embodiments, one or more aspects of the non-transitory memory 206 may include remotely accessible networked storage devices in a cloud computing configuration. As an example, the non-transitory memory 206 may be part of a Picture Archiving and Communication System (PACS) configured to store, for example, patient histories, imaging data, test results, diagnostic information, administrative information, and / or scheduling information.

[0040] The image processing system 200 may also include a user input device 232. The user input device 232 may include one or more of a touch screen, keyboard, mouse, touchpad, motion-sensing camera, or other devices configured to enable a user to interact with and manipulate the data stored within the image processor 231.

[0041] The display device 233 may include one or more display devices utilizing any type of display technology. In some embodiments, the display device 233 may include a computer monitor and may display unprocessed images, processed images, parametric maps, and / or examination reports. The display device 233 may be combined with the processor 204, the non-transitory memory 206, and / or the user input device 232 in a shared housing or may be a peripheral display device. The display device 233 may include a monitor, touch screen, projector, or another type of display device that enables a user to view medical images and / or interact with the various data stored in the non-transitory memory 206. In some embodiments, the display device 233 may be included in a smart phone, tablet, smart watch, etc.

[0042] It will be understood that Figure 2 the illustrated medical image processing system 200 is a non-limiting embodiment of an image processing system, and other imaging processing systems may include more, fewer, or different components without departing from the scope of the present disclosure. Additionally, in some embodiments, at least a portion of the medical image processing system 200 may be included in Figure 1in the ultrasonic imaging system 100, and vice versa (e.g., at least a part of the ultrasonic imaging system 100 may be included in the medical image processing system 200).

[0043] Figure 3 illustrates Figure 1 an embodiment of the ultrasonic probe 106 of the ultrasonic imaging system 100. The ultrasonic probe 106 includes a housing 300 that can accommodate transducer elements 104 ( Figure 3 not shown in the figure). The housing 300 can contact the imaging subject (e.g., a patient) along the facing surface 302 of the housing 300. The facing surface 302 of the housing 300 may have a generally rectangular shape that is elongated along a first edge 304 relative to a second perpendicular edge 306.

[0044] The ultrasonic probe 106 can be moved along the outer surface of the imaging subject (e.g., in a sweeping motion) to acquire ultrasonic image data of the internal structure of the imaging subject. For example, the imaging subject can be a patient, and the ultrasonic probe 106 can be moved over the patient's chest to acquire ultrasonic image data of the patient's lungs and ribs. A user (such as an operator of the ultrasonic imaging system 100) can physically sweep the ultrasonic probe 106 through multiple intercostal spaces of the patient's chest while acquiring a 3D data set. In some embodiments, a dynamic 3D ultrasonic image can be gradually generated in real time while the user sweeps the ultrasonic probe 106. Further details regarding the acquisition of ultrasonic image data of the lungs and ribs are described with respect to Figure 4 The generation of the dynamic 3D ultrasonic image is described with respect to Figures 5 to 10 Ultrasonic signals can be pulsed from the transducer elements 104 into the body of the subject, and the pulsed ultrasonic signals are backscattered from structures inside the body to produce echoes that return to the transducer elements 104 housed in the housing 300. As described with respect to Figure 1 the ultrasonic probe 106 can be a 1.5D probe, a 3D probe, or any ultrasonic probe capable of performing live 3D imaging, such as a matrix array probe.

[0045] As used herein, the terms “system” and “module” may include a hardware and / or software system that operates to perform one or more functions. For example, a module or system may include a computer processor, a controller, or other logic-based devices that perform operations based on instructions stored on a tangible and non-transitory computer-readable storage medium (such as computer memory), or may be included therein. Alternatively, a module or system may include a hardwired device that performs operations based on the hardwired logic of the device. The various modules or systems shown in the drawings may represent hardware that operates based on software or hardwired instructions, software that instructs the hardware to perform operations, or a combination thereof.

[0046] A "system" or "module" may include or represent hardware and associated instructions (e.g., software stored on a tangible and non-transitory computer-readable storage medium such as a computer hard drive, ROM, RAM, etc.) that perform one or more operations described herein. The hardware may include an electronic circuit that includes and / or is connected to one or more logic-based devices such as a microprocessor, a processor, or a controller, etc. These devices may be off-the-shelf devices that are appropriately programmed or instructed to perform the operations described herein according to the instructions described above. Additionally or alternatively, one or more of these devices may be hardwired to logic circuitry to perform these operations.

[0047] Figure 4 Illustrated is the thorax 400 of patient 404 according to an embodiment. Ultrasonic image data acquired by an ultrasonic probe 106 (e.g., Figure 1 and Figure 3 of) and displayed as a dynamic 3D ultrasonic image may represent portions of the thorax 400 that include the lungs 408 of patient 404, a plurality of ribs, and the sternum 410. From the bottom 442 to the top 440 of the thorax 400, the plurality of ribs may include: a first rib 421, a second rib 422, a third rib 423, a fourth rib 424, a fifth rib 425, a sixth rib 426, a seventh rib 427, an eighth rib 428, a ninth rib 429, and a tenth rib 430. Figure 4 Also shown are a plurality of intercostal spaces located between these ribs, which are also referred to herein as rib spaces. For example, a first intercostal space 431, a second intercostal space 432, a third intercostal space 433, a fourth intercostal space 434, and a fifth intercostal space 435 are represented in Figure 4 . The first intercostal space 431 is located between the first rib 421 and the second rib 422, the second intercostal space 432 is located between the second rib 422 and the third rib 423, the third intercostal space 433 is located between the third rib 423 and the fourth rib 424, the fourth intercostal space 434 is located between the fourth rib 424 and the fifth rib 425, and the fifth intercostal space 435 is located between the fifth rib 425 and the sixth rib 426. The thorax 400 includes Figure 4 additional intercostal spaces not specifically identified in

[0048] Figures 1 to 3 The ultrasonic probe 106 of may generally be held by a user in an orientation in which the edge 306 of the outer housing 300 is oriented parallel to the plurality of ribs (e.g., more parallel to the plurality of ribs than perpendicular to the plurality of ribs). Figure 1 and Figure 3The ultrasound probe 106 can be moved transversely to the direction in which multiple ribs extend. For example, the ultrasound probe 106 can be moved along a direction substantially parallel to the longitudinal division plane 402. The ultrasound probe 106 can be moved along the exterior of the patient 404 in a direction more parallel to the longitudinal division plane 402 than perpendicular to it. This orientation of the ultrasound probe 106 can be referred to as the longitudinal division position or orientation. Alternatively, the ultrasound probe 106 can be held by the user in an orientation perpendicular to the longitudinal division orientation such that the edge 306 is perpendicular to multiple ribs while the ultrasound probe 106 is moved along a direction substantially parallel to the longitudinal division plane 402. This orientation can be referred to as the transverse position or orientation of the ultrasound probe 106.

[0049] Figure 5 A flowchart of a method 500 for generating dynamic 3D ultrasound images is shown. Specifically, the method 500 provides a workflow for generating dynamic 3D ultrasound images of the ribs, intercostal spaces, and other anatomical structures of the thoracic cavity of an imaging subject. For example, the method 500 can be a summary method for generating dynamic images of elements of the thoracic cavity 400 and intercostal spaces (such as the first intercostal space 431, the second intercostal space 432, etc.) of the patient 404 described with respect to Figure 4 The method 500 will be described for dynamic 3D ultrasound images acquired using an ultrasound imaging system (such as Figure 1 the ultrasound imaging system 100), but other ultrasound imaging systems can also be used. In addition, the method 500 can be adapted to other imaging modalities. The method 500 can be implemented by one or more of the above systems, which include Figure 1 the ultrasound imaging system 100, Figure 2 the medical image processing system 200, and / or Figure 3 the ultrasound probe 106, as further described herein. In short, the instructions for performing the method 500 can be stored as computer-readable instructions in a non-transitory memory (such as Figure 1 the memory 120 and / or Figure 2 the non-transitory memory 206) and executed by a processor (such as Figure 1 the processor 116 and / or Figure 2 the processor 204). In addition, in some embodiments, the method 500 is executed in real time (e.g., when 3D ultrasound image data is captured) (e.g., using an ultrasound probe (such as Figure 3of the ultrasound probe 106)). For example, when using an ultrasound probe to capture dynamic 3D ultrasound image data, method 500 may perform the operations described herein (e.g., generating multiple 3D rib space segments, etc.). In other embodiments, at least a portion of method 500 is performed offline after the imaging scan that is executed to capture 3D ultrasound image data. For example, even when the ultrasound system is not actively operating to acquire 3D ultrasound image data, the processor may evaluate the 3D ultrasound image data stored in the memory.

[0050] At 502, method 500 includes acquiring dynamic 3D ultrasound image data, also referred to herein as "image data". As described above, 3D ultrasound image data may be acquired in real-time or near real-time. In another embodiment, dynamic 3D ultrasound image data may be captured during an imaging scan and stored in a memory and / or database for later use, and acquiring the dynamic 3D ultrasound data may include requesting and receiving the dynamic 3D ultrasound data from the database. A 3D ultrasound probe (such as the ultrasound probe 106 described with respect to Figure 1 and Figure 3 may be used to acquire the 3D ultrasound image data. As described above, the imaging scan for capturing 3D ultrasound image data may include a longitudinal scan, an oblique scan, and / or a panoramic sweep. In a longitudinal scan, the ultrasound probe is positioned perpendicular to the ribs. In an oblique scan, the ultrasound probe is positioned along the intercostal space between the ribs. In a panoramic sweep, the ultrasound probe may be (e.g., by the user) swept from the cephalic side to the caudal side of the thoracic cavity along the midsagittal plane, as described with respect to Figure 4 The user (such as the operator of the ultrasound imaging system 100) may physically sweep the ultrasound probe 106 through multiple rib spaces of the patient's thoracic cavity while acquiring the 3D data set. In some embodiments, a dynamic 3D ultrasound image may be gradually generated in real-time as the user sweeps the ultrasound probe 106, as further described herein. As described herein, the preferred sweep motion for capturing image data of the rib space is a midsagittal motion from the top rib (e.g., the tenth rib 430) downward to the bottom rib (e.g., the first rib 421) or from the bottom rib upward to the top rib. In some embodiments, the ultrasound probe 106 may be swept through a portion of the thoracic cavity 400, e.g., in a midsagittal motion from the bottom 442 to the top 440 of the thoracic cavity 400 through multiple rib spaces and vice versa. As further described herein, multiple views from the imaging scan may be stitched together to provide the anatomical and spatial relationships of the rib space segments captured in the dynamic 3D ultrasound image data.

[0051] A 3D ultrasound probe can continuously capture 3D ultrasound image data, and the characteristics of the 3D ultrasound image data can change during an imaging scan. For example, the structure of the pulmonary pleura can change with the respiratory cycle, movement, etc. of the imaging subject. The acquired 3D ultrasound image data is dynamic image data, such as a video showing the movement of one or more portions of the intercostal spaces and / or rib shadows. Such movement can cause one or more features of interest to sometimes appear and disappear from the 3D ultrasound image data at other times. For example, B-lines or other features in the image data that may indicate pneumonia infection, air bronchogram, or other injuries may be visible in some but not all image frames.

[0052] As described above, dynamic 3D ultrasound image data can be captured by sweeping the ultrasound probe through the thoracic cavity of the imaging subject. During the imaging scan, the ultrasound probe can be moved over different regions of the thoracic cavity such that different amounts of image data are captured for different regions of the thoracic cavity. For example, the ultrasound probe can be swept through the first rib and the first intercostal space, the second rib and the second intercostal space, and the third rib and the third intercostal space. Then, the ultrasound probe can be positioned above the third rib and / or the third intercostal space. The ultrasound probe can be further swept downward along the thoracic cavity (e.g., parallel to the longitudinal division plane 402), and / or the ultrasound probe can be moved back toward the top of the thoracic cavity to capture additional image data of the first rib, the first intercostal space, etc. In this example, the acquired dynamic 3D ultrasound image data can be a continuous video, e.g., which starts at the top of the thoracic cavity, moves downward, pauses in the middle of the thoracic cavity, moves back upward, continues downward, etc. to capture image data. Thus, more image data can be captured for some regions / structures compared to other regions / structures.

[0053] A live view of the 3D ultrasound image data (e.g., the volumetric structure of the internal structure of the imaging subject) can show the structures within the FOV of the ultrasound probe at the current position of the ultrasound probe. The size of the FOV of the ultrasound probe used to capture image data of a patient's thoracic cavity can be set to capture one to two intercostal spaces at a time. It may be desirable to view more structures than those shown in the FOV (e.g., more current ribs, intercostal spaces on either side, ribs on either side, etc.), which may assist in diagnostic analysis. Currently, visualizing additional regions requires moving the ultrasound probe to capture additional regions within the FOV of the ultrasound probe. However, it may be desirable to view multiple regions simultaneously, including regions that may not be viewable simultaneously within the size limitations of the FOV of the ultrasound probe. Figure 7 An example of a 3D ultrasound image captured within the FOV of the ultrasound probe is shown.

[0054] At 504, method 500 includes generating a plurality of 3D rib interspace segments from dynamic 3D ultrasound image data. Each of the plurality of rib interspace segments can be a region of interest that includes a rib interspace (e.g., the space between two ribs) and at least a portion of each rib on either side of the rib interspace. As described above, 3D ultrasound image data obtained sequentially and / or stored during an imaging scan can be collected, and the sequence can be the same as or different from the anatomical sequence of the imaged subject. In some embodiments, each of the plurality of 3D rib interspace segments can be generated in real time as the ultrasound probe 106 is swept by a user across a corresponding rib interspace. Additionally, the FOV of the ultrasound probe can capture more than one rib interspace and its boundary ribs. One or more methods can be used to identify the individual rib interspace segments in the 3D ultrasound image data. For example, a processor can be configured to automatically examine the characteristics of pixels and / or other subsets of the image data to identify the rib interspace segments, such as the color, intensity, or brightness of the pixels in the image data. The processor can further divide the dynamic 3D ultrasound image data into a plurality of 3D rib interspace segments based on the characteristics of different structures (e.g., the curvature of the ribs, the width of the rib interspace). In another example, the center of the FOV of the ultrasound probe can be tracked during an imaging scan, and the relative position of the ultrasound probe on the surface of the thorax with respect to structures (e.g., ribs and tear spaces) can be used to automatically identify the region of interest. With respect to Figure 6 Method 600 describes further details regarding the generation of rib interspace segments. Figure 7 An example of an annotated dynamic 3D ultrasound image is shown in

[0055] At 506, method 500 includes generating a dynamic panoramic 3D ultrasound image depicting a plurality of 3D rib interspace segments in anatomical order. Characteristics of the dynamic 3D ultrasound image data used to generate the plurality of 3D rib interspace segments can be further used to identify specific structures shown in each 3D rib interspace segment. For example, the curvature of the ribs, the width of the rib interspaces, the relative position of the ultrasound probe, and / or the structures captured in the FOV of the ultrasound probe can be automatically examined to identify which rib and / or rib interspace of the thoracic cavity is shown in each 3D rib interspace segment (e.g., first rib 421, first rib interspace 431, second rib 422, etc.). The plurality of 3D rib interspace segments can be mapped to a reference thoracic cavity or other anatomical reference to order each of the plurality of 3D rib interspace segments in anatomical order. As briefly described above, the dynamic 3D ultrasound image data of the ribs and rib interspaces can be captured in an order different from the anatomical order. Accordingly, identifying the specific rib and rib interspace structures in the plurality of 3D rib interspace segments and ordering the plurality of 3D rib interspace segments presents the dynamic 3D ultrasound image data in a configuration adapted to the anatomical structure. Depending on the time point and the perspective of the dynamic 3D ultrasound image data, this can enable visualization of irregularities and / or lesions that extend between / across the plurality of rib interspaces and / or that are variably visible. Once the plurality of 3D rib interspace segments are positioned in anatomical order, the segmented 3D rib interspaces can be joined (e.g., stitched) to form a single dynamic 3D ultrasound image in which the plurality of 3D rib interspace segments are contiguous (e.g., there are no gaps or breaks between them other than the rib interspaces of the anatomical structure).

[0056] At 508, method 500 includes temporally synchronizing a plurality of dynamic 3D rib interspace segments. As described above, the dynamic 3D ultrasound image data for each 3D rib interspace segment may have different durations (e.g., different periods), and / or may include different numbers of image frames. Generating a dynamic 3D ultrasound image includes temporally synchronizing the plurality of dynamic 3D rib interspace segments such that the dynamic 3D ultrasound image has a single duration that synchronizes the dynamic 3D rib interspace segments. Temporal scaling may be applied to one or more of the dynamic 3D rib interspace segments to expand or contract one or more of the dynamic 3D rib interspace segments. For example, a first dynamic 3D rib interspace segment may be expanded in time such that the amount of time for which its dynamic 3D image data is played is longer; and a second dynamic 3D rib interspace segment may be contracted in time such that the amount of time for which its dynamic 3D image data is played is shorter. After temporal scaling, the durations of the first dynamic 3D rib interspace segment and the second dynamic 3D rib interspace segment may be equal. Expanding a dynamic 3D rib interspace segment in time causes the associated dynamic 3D ultrasound image data to be played at a frame rate slower than the acquisition frame rate, and contracting a dynamic 3D rib interspace segment in time causes the associated dynamic 3D ultrasound image data to be played at a frame rate faster than the acquisition frame rate.

[0057] In addition to, or instead of, contracting or expanding the dynamic 3D ultrasound image data for one or more of the dynamic 3D rib interspace segments in time, the processor may also utilize additional techniques to perform its temporal scaling. For example, if more dynamic 3D ultrasound image data is captured for a first dynamic 3D rib interspace segment than for a second dynamic 3D rib interspace segment, the dynamic 3D ultrasound image data for the second dynamic 3D rib interspace segment may have fewer frames than the dynamic 3D ultrasound image data for the first dynamic 3D rib interspace segment. Temporal scaling may include playing the dynamic 3D ultrasound image data for a dynamic 3D rib interspace segment having fewer frames more than once, while the dynamic 3D ultrasound image data for a dynamic 3D rib interspace segment having more frames may be played once. For example, if a first dynamic 3D rib interspace segment has ten frames and a second dynamic 3D rib interspace segment has thirty frames, the processor may play the dynamic 3D ultrasound image data for the first dynamic 3D rib interspace segment three times and play the dynamic 3D ultrasound image data for the second dynamic 3D rib interspace segment once to provide appropriate temporal scaling for each dynamic 3D rib interspace segment.

[0058] At 510, method 500 includes outputting a dynamic panoramic 3D ultrasound image for display. In some examples, the display is included in an ultrasound imaging system (such as display device 118).Figures 8 to 10 An example display of a dynamic panoramic 3D ultrasound image is shown. In some examples, the dynamic panoramic 3D ultrasound image can be saved with and / or without annotations (e.g., pleural line indication identification). Additionally, at least in some examples, the original unprocessed ultrasound data can be saved. The memory can be local to the ultrasound imaging system or can be remote memory. For example, unannotated images and annotated images (e.g., as structured reports in a PACS system) can be saved and / or archived such that they can be retrieved and used to generate a formal doctor-signed report that can be included in the patient's medical record. Thus, generating a dynamic panoramic 3D ultrasound image can simplify image analysis by showing the detailed topography of the pleura in one image (e.g., the dynamic panoramic 3D ultrasound image) rather than in multiple dynamic 3D ultrasound images.

[0059] Figure 6 A flowchart of a method 600 for generating a dynamic 3D ultrasound image from 3D ultrasound image data is shown. Method 600 provides further details and can be an Figure 5 embodiment of method 500. At least in part with respect to Figure 5 and with reference to Figures 7 to 10 Method 600 will be described, as will be further described after the description of method 600. Method 600 will be described for dynamic 3D ultrasound images acquired using an ultrasound imaging system (such as Figure 1 ultrasound imaging system 100), but other ultrasound imaging systems can also be used. Additionally, method 600 can be adapted to other imaging modalities. Method 600 can be implemented by one or more of the systems described above, which include Figure 1 ultrasound imaging system 100, Figure 2 medical image processing system 200, and / or Figure 3 ultrasound probe 106, as further described herein. Briefly, the instructions for performing method 600 can be stored as computer-readable instructions in non-transitory memory (such as Figure 1 memory 120 and / or Figure 2 non-transitory memory 206) and executed by a processor (such as Figure 1 processor 116 and / or Figure 2 processor 204). Additionally, in some embodiments, method 600 is executed in real time (e.g., when 3D ultrasound image data is captured) (e.g., using an ultrasound probe (such as Figure 3of the ultrasound probe 106)). For example, when using an ultrasound probe to capture dynamic 3D ultrasound image data, method 600 may perform the operations described herein (e.g., generating multiple 3D rib interspace segments, etc.). In other embodiments, at least a portion of method 600 is performed offline after the imaging scan that is performed to capture 3D ultrasound image data. For example, even when the ultrasound system is not actively operated to acquire 3D ultrasound image data, the processor may evaluate the 3D ultrasound image data stored in the memory.

[0060] At 602, method 600 includes acquiring dynamic 3D ultrasound image data, also referred to herein as "image data". As described with respect to operation 502 of method 500, dynamic 3D ultrasound image data may be acquired. Briefly, 3D ultrasound image data may be acquired in real time, near real time, or 3D ultrasound image data may be captured during an imaging scan and stored in a memory and / or database for later use, and acquiring dynamic 3D ultrasound data may include requesting and receiving dynamic 3D ultrasound data from the database. A 3D ultrasound probe (such as the ultrasound probe 106 described with respect to Figure 1 and Figure 3 may be used to acquire 3D ultrasound image data.

[0061] At 604, method 600 includes identifying multiple rib interspaces of the dynamic 3D ultrasound image data. As described with respect to Figure 5 operation 504 of method 500, method 500 includes generating multiple 3D rib interspace segments from the dynamic 3D ultrasound image data. Operation 604 includes identifying multiple rib interspaces, which may include separating the dynamic 3D ultrasound image data into multiple rib interspace segments (e.g., where each rib interspace segment of the multiple rib interspace segments includes a rib interspace of the multiple rib interspaces). As described with respect to method 500, the FOV of the ultrasound probe may simultaneously capture more than one rib interspace and its boundary ribs. One or more methods may be used to identify the individual rib interspace segments in the 3D ultrasound image data. For example, the processor may be configured to automatically examine the characteristics of pixels and / or other subsets of the image data to identify rib interspace segments, such as the color, intensity, or brightness of the pixels in the image data. The processor may further divide the dynamic 3D ultrasound image data into multiple 3D rib interspace segments based on the characteristics of different structures (e.g., the curvature of the ribs, the width of the rib interspaces). As another example, the center of the FOV of the ultrasound probe may be tracked during the imaging scan, and the relative position of the ultrasound probe on the surface of the chest cavity with respect to structures (e.g., ribs and tear gaps) may be used to automatically identify the region of interest.

[0062] 3D ultrasound image data includes a plurality of 3D ultrasound image data frames, each frame being acquired at a different time. As long as the ultrasound probe is translated during the acquisition of the ultrasound image data, each image frame within the image frames can be acquired from different spatial positions relative to the anatomical structure of the imaging subject. The processor can be further configured to identify a timestamp or other identifier associated with the transition from an intercostal space to a rib shadow in the 3D ultrasound image data. For example, the intensity associated with the rib shadow may be relatively low, while the intensity associated with the intercostal space is relatively high. When the ultrasound probe is translated during the acquisition of the 3D ultrasound image data, the anatomical structure acquired in each frame is different. The processor can be configured to identify a frame in the 3D ultrasound image data in which the 3D ultrasound image data transitions from a relatively low intensity (e.g., associated with a rib shadow) to a relatively high intensity (e.g., associated with a particular intercostal space). Additionally, the processor can be configured to identify a frame in the 3D ultrasound image data in which the 3D ultrasound image data transitions from a particular intercostal space having a relatively high intensity to an adjacent (e.g., next) rib shadow having a relatively low intensity.

[0063] At 606, method 600 includes identifying and exposing the pleural surface of each of a plurality of intercostal spaces. In an inflated lung, the pleura that forms the outer boundary of the lung against the chest wall can provide an anatomical lung structure that is substantially detectable by ultrasound. The pleura can appear in a 3D ultrasound image as a hyperechoic level segment of brighter (e.g., whiter) pixels (referred to as the pleural line), which moves synchronously with the respiratory cycle in a phenomenon called pleural sliding. The 3D pleural surface includes the volumetric topography of the pulmonary pleura. The pleural topography can be visualized in the 3D pleural surface, which can assist in identifying and / or diagnosing lung conditions that can be visualized as irregular portions in the pleural surface topography. The 3D pleural surface can show the pleural topography in a wider area of the lung compared to what can be captured in a 2D ultrasound image. Additionally, in a dynamic 3D ultrasound image, the changes in the 3D pleural surface over time can be shown, where these changes reflect changes in the patient's anatomical structure visualized using the ultrasound probe (e.g., due to patient breathing, movement, etc.).

[0064] Detecting the bottom edge of the pleura (e.g., where the pleural surface ends and the interior of the lung begins) can include identifying the lower and upper boundaries of the pleura based on the difference in brightness between pixels, such as by using edge detection techniques or gradient changes. For example, the processor can apply an edge detection algorithm, which can be included in Figure 2In the 3D generation module 212. The edge detection algorithm may include one or more mathematical models that are used to identify points (e.g., pixels) where the image brightness changes abruptly and / or has interrupted portions to identify the lower and upper boundaries of the pleural line. As an example, the processor may apply the edge detection algorithm to the region with the highest local variation. As another example, additionally or alternatively, a gradient algorithm may identify the local maximum and / or minimum pixel brightness at the pleural position to identify the lower and upper boundaries of the pleural line in each image. In some embodiments, a deep learning model trained for real-time or static (e.g., non-real-time) pleural detection is used to identify the bottom edge of the pleura.

[0065] The lower and upper boundaries of the pleura may include subpleural consolidation. For example, the bottom edge (e.g., as further described herein, the boundary indicating the removal of ultrasound data below it) may be positioned a predetermined distance from the pleural line towards the lung. It can be understood that conventionally, subpleural consolidation extends a maximum distance 'n' from the pleural surface towards the lung. Thus, the bottom edge of the pleura may be positioned a distance 'n' from the lower boundary of the pleura identified as described above. In other embodiments, the same or different edge detection algorithms as described above may be used to identify subpleural consolidation. Relative to Figure 7 An example 3D ultrasound image including an indication of the pleural line is described.

[0066] After identifying the bottom edge of the pleural line, operation 606 of method 600 includes removing ultrasound data below the bottom edge of the pleural line from each 3D ultrasound image of the 3D ultrasound image data. As described relative to Figure 7 The ultrasound image data vertically below the bottom edge of the pleural line may be noise or other imaging artifacts that do not indicate the topography of the pleural line. As described above, subpleural consolidation is included in the boundary defined by the bottom edge. Removing the ultrasound data may include generating a new image data set for each 3D ultrasound image, the new image data set including the ultrasound data above and including the pleural line and not including the data below the bottom edge of the pleural line. Thus, the 3D pleural surface is exposed in the 3D ultrasound image formed by the new image data set. The operations 604 for detecting and exposing the pleural surface of the lung region in the 3D ultrasound image data may be performed for all the plurality of intercostal spaces in the 3D ultrasound image data. In this way, the 3D pleural surface is exposed for each 3D ultrasound image of each intercostal space.

[0067] At 608, method 600 includes using image characteristics of the pleural surface to sequentially align multiple intercostal spaces according to anatomical order. Characteristics of the structures in each of the multiple intercostal spaces (e.g., curvature of the rib, width of the intercostal space) can be used to identify the ribs and / or intercostal spaces shown in each of the multiple intercostal spaces. As described above, the multiple intercostal spaces can be mapped to a reference thoracic cavity or other anatomical reference to sort each of the multiple intercostal spaces according to anatomical order.

[0068] At 610, method 600 includes synchronizing the display timing of each of the multiple intercostal spaces such that the display start time and end time of each intercostal space occur simultaneously. The 3D ultrasound image data is dynamic 3D ultrasound image data, such as a video showing movement of one or more portions of the intercostal spaces and / or ribs. Multiple sets of 3D ultrasound image data for different regions of interest may include 3D ultrasound image data of different durations. Synchronizing the 3D ultrasound data in time includes identifying a target duration to be used in the dynamic 3D ultrasound image. For example, a processor may determine the acquisition duration of each region of interest in the 3D ultrasound image data and use the acquisition duration to identify the target duration. For example, intensity information from the 3D ultrasound image data can be used to identify the start time and end time when an interesting structure exists in the region of interest in the 3D ultrasound image data. As described in the corresponding method 500, the 3D ultrasound image data for one or more intercostal spaces can be expanded or contracted in time.

[0069] As another example, synchronizing the display timing of each intercostal space includes adjusting the display playback speed of one or more intercostal spaces. Adjusting the playback speed may include adjusting the amount of time for each frame of the dynamic 3D ultrasound image data showing a given intercostal space, thereby adjusting the overall playback speed for the given intercostal space. The playback speed can be adjusted by playing each frame of the dynamic 3D ultrasound image data of a first intercostal space with fewer frames over a longer duration and playing each frame of the dynamic 3D ultrasound image data of a second intercostal space segment with more frames over a shorter duration. For example, the playback duration of each frame of a first intercostal space with ten frames can be three times that of each frame of a second intercostal space with thirty frames (e.g., the former's duration is three times that of the latter) to provide appropriate temporal scaling for each intercostal space.

[0070] In another example, synchronizing display timing includes: identifying a respiratory cycle in each of a plurality of rib interspaces; and synchronizing the respiratory cycles of the plurality of rib interspaces. As described above, segments of dynamic 3D ultrasound image data may show different regions, structures, and / or abnormalities at different times during an imaging scan (e.g., at different parts of a respiratory cycle). Thus, it is desirable to synchronize the display timing of the plurality of rib interspaces by synchronizing the respiratory cycles of the plurality of rib interspaces such that structures, lesions, and / or configurations that occur simultaneously in the imaged subject's anatomy are shown simultaneously in a single dynamic 3D ultrasound image. Synchronizing the respiratory cycles of the plurality of rib interspaces includes: simultaneously showing dynamic 3D ultrasound image data captured during a first inspiration over a first duration; and simultaneously showing dynamic 3D ultrasound image data captured during a first expiration over a second duration separate from the first duration. For example, a respiratory cycle may include a series of alternating inspirations and expirations. Image characteristics (such as the color, intensity, brightness, etc. of pixels in the image data) may be used to identify a part of the respiratory cycle (e.g., inspiration or expiration) in the dynamic 3D ultrasound data. For example, due to the expansion of lung tissue, pixels on the pleural surface may be brighter during inspiration compared to the brightness of pixels on the pleural surface during expiration. The dynamic 3D ultrasound image data for each rib interspace may be synchronized such that the start time and end time of the display of each rib interspace occur simultaneously. This may include adjusting the playback speed of one or more rib interspaces, and / or expanding and / or contracting the 3D ultrasound image data for one or more rib interspaces in time.

[0071] At 612, method 600 includes displaying 3D ultrasound image data as a dynamic 3D ultrasound image. In some examples, displaying the dynamic 3D ultrasound image on a display (such as display device 118) included in an ultrasound imaging system. Figures 8 to 10 An example display of a dynamic panoramic 3D ultrasound image is shown. In some examples, the dynamic panoramic 3D ultrasound image may be saved with and / or without annotations (e.g., pleural line indication identification). Additionally, at least in some examples, the original unprocessed ultrasound data may be saved. The memory may be local to the ultrasound imaging system or may be remote memory. For example, unannotated images and annotated images (e.g., as a structured report in a PACS system) may be saved and / or archived such that they can be retrieved and used to generate a formal doctor-signed report that may be included in the patient's medical record. Thus, generating a dynamic panoramic 3D ultrasound image may simplify image analysis by showing the detailed topography of the pleura in one image (e.g., the dynamic panoramic 3D ultrasound image) rather than in multiple dynamic 3D ultrasound images.

[0072] Figure 7An example annotated 3D rib interspace segment 702 that can be used to generate a 3D pleural surface image 704 is shown. Each of the annotated 3D rib interspace segment 702 and the 3D pleural surface image 704 is described herein with reference to Figures 1 to 6 and includes the lungs 408, ribs (e.g., first rib 421), and rib interspaces (e.g., first rib interspace 431) of a patient 404 acquired using an ultrasound probe 106 held in a longitudinal split orientation (e.g., performing a panoramic sweep). The rib shadows indicate the positions where the pulsed ultrasound signals are blocked by the ribs.

[0073] The annotated 3D rib interspace segment 702 includes a pleural line indication identifier 706 of the pleural line. The pleural line indication identifier 706 can be positioned on the bottom edge (e.g., lower boundary) of the pleural line and can visually indicate the pixels in the annotated 3D rib interspace segment 702 that are identified as the pleural line. For example, the pleural line is indicated by the pleural line indication identifier 706. In other examples, the pleural line indication identifier 706 can be depicted (e.g., with a continuous line) to visually indicate the pleural line in the annotated 3D rib interspace segment 702. The vertical position of each pleural line indication identifier in the pleural line indication identifier 706 can be different and can reflect the curvature, protrusions, cavities, and / or other irregularities in the pleural line. For example, the first pleural indication identifier can have a higher vertical position relative to the second pleural indication identifier. The ultrasound image data vertically below the pleural line (e.g., below each pleural line indication identifier in the pleural line indication identifier 706 and the gaps between them) can be noise and thus may not indicate the topography of the pleural line. For example, a protrusion 718 extending beyond (e.g., below) the pleural line indication identifier 706 can be an abnormality of the pleural surface 716 and can indicate a lesion. As described with respect to Figure 6 method 600, the 3D pleural surface image 704 shows a 3D pleural surface 716 that is generated from the annotated 3D rib interspace segment 702 by removing the dynamic 3D ultrasound image data below the pleural line indicated by the pleural line indication identifier 706.

[0074] Figure 8 A first example dynamic panoramic 3D ultrasound image 800 is shown. The dynamic panoramic 3D ultrasound image 800 includes a first dynamic rib interspace segment 802, a second dynamic rib interspace segment 804, a third dynamic rib interspace segment 806, and a fourth dynamic rib interspace segment 808. For illustrative purposes, each of these dynamic rib interspace segments is illustrated as Figure 7The 3D pleural surface image 704. It should be understood that, in fact, each of these dynamic rib interspace segments may show different topographies and structures of the thoracic cavity. The dynamic panoramic 3D ultrasound image 800 also includes the rib interspaces between each of these dynamic rib interspace segments. The first rib shadow 812 is formed, for example, by the first rib between the first dynamic rib interspace segment 802 and the second dynamic rib interspace segment 804. The second rib shadow 814 is formed, for example, by the second rib between the second dynamic rib interspace segment 804 and the third dynamic rib interspace segment 806. The third rib shadow 816 is formed, for example, by the third rib between the third dynamic rib interspace segment 806 and the fourth dynamic rib interspace segment 808.

[0075] Figure 9 Shows a second example of a dynamic 3D ultrasound image 900. The second example dynamic 3D ultrasound image 900 can be Figure 8 A perspective view of the first example dynamic panoramic 3D ultrasound image 800. Additionally, the dynamic 3D ultrasound image 900 shows an exemplary topography of the pleural surface 716 of the fourth dynamic rib interspace segment 808 and the third dynamic rib interspace segment 806, where the third rib shadow 816 is formed between the fourth dynamic rib interspace segment and the third dynamic rib interspace segment. The pleural surface 716 of the third dynamic rib interspace segment 806 includes a protrusion 718.

[0076] Figure 10 Shows an example display 1001 including Figure 9 The dynamic 3D ultrasound image 900. The display 1001 can be, for example, Figure 1 The display device 118. The display 1001 includes sliders for adjusting the brightness (e.g., brightness slider 1010), color / hue (e.g., hue box 1012), and position (e.g., coordinate selector 1014) of one or more light sources, as well as a reference axis system 1090. In one example, the y-axis can be the vertical axis (e.g., parallel to the gravity axis), the x-axis can be the transverse axis (e.g., the horizontal axis), and the z-axis can be the longitudinal axis. However, in other examples, these axes may have other orientations.

[0077] Methods 500 and 600 optionally include applying shading to a 3D pleural surface when outputting a dynamic 3D ultrasound image for display. The 3D pleural surface can be shaded via at least one virtual light source 1020 positioned as if on the medial side of the lung. The shading can highlight protrusions (e.g., protrusion 718) extending from the 3D pleural surface toward the medial side of the lung. For example, at least one virtual light source can be positioned at a preset location within the medial side of the lung, such as at an acute angle relative to the 3D pleural surface and / or a protrusion protruding from the 3D pleural surface. The position, brightness, color, and / or number of the virtual light sources can be adjusted in response to receiving user input.

[0078] In this way, the processor can automatically generate a dynamic 3D ultrasound image including multiple intercostal spaces and rib shadows. The methods and systems described herein enable visualization of the entire pleural surface in a single image, where the pleural surface can be captured by an ultrasound probe through multiple separate FOV captures. This can eliminate the diagnostic step of searching for a specific view of the pleural surface. In this way, a wide range of ultrasound findings and lesions can be visualized, including pleural irregularities, pneumothorax, and viral and bacterial infections. Therefore, the amount of time spent by healthcare professionals viewing medical images can be reduced, enabling healthcare professionals to focus on patient care and comfort. In addition, by automatically arranging multiple intercostal spaces and intercostal space segments in anatomical order, irregularities can be displayed in an anatomically relevant context to further simplify the diagnostic process.

[0079] The technical effect of generating a dynamic 3D ultrasound image is that, due to a reduction in the number of additional imaging scans attributable to increased accuracy and detail in the rendering of the pleural surface image, the processing power used by the imaging system can be reduced.

[0080] The present disclosure also provides support for a method for ultrasonic imaging, the method comprising: acquiring dynamic three-dimensional ultrasonic image data; generating a plurality of dynamic three-dimensional rib interspace segments from the dynamic three-dimensional ultrasonic image data; generating a dynamic panoramic three-dimensional ultrasonic image depicting the plurality of dynamic three-dimensional rib interspace segments in anatomical order; synchronizing the plurality of dynamic three-dimensional rib interspace segments in time; and outputting the dynamic panoramic three-dimensional ultrasonic image for display. In a first example of the method, generating the plurality of dynamic three-dimensional rib interspace segments comprises: automatically identifying a plurality of rib shadows in the dynamic three-dimensional ultrasonic image data; using the plurality of rib shadows to define boundaries between each of the plurality of rib interspaces; and segmenting the dynamic three-dimensional ultrasonic image data into a plurality of volumes based on the boundaries between each of the plurality of rib interspaces. In a second example of the method, optionally including the first example, the plurality of rib shadows are detected by tracking the movement of an ultrasonic probe used to capture the dynamic three-dimensional ultrasonic image data during an imaging scan. In a third example of the method, optionally including one or both of the first example and the second example, the plurality of rib shadows are detected based on the intensity and / or brightness of pixels of the dynamic three-dimensional ultrasonic image data. In a fourth example of the method, optionally including one or more or each of the first example to the third example, depicting the plurality of dynamic three-dimensional rib interspace segments in the anatomical order comprises: identifying and exposing the pleural surface of each three-dimensional rib interspace segment; and sorting the plurality of dynamic three-dimensional rib interspace segments in the anatomical order based on the pleural surface. In a fifth example of the method, optionally including one or more or each of the first example to the fourth example, synchronizing the plurality of dynamic three-dimensional rib interspace segments in time comprises: applying time scaling to expand and / or contract one or more of the plurality of dynamic three-dimensional rib interspace segments. In a sixth example of the method, optionally including one or more or each of the first example to the fifth example, the dynamic three-dimensional ultrasonic image data is acquired by sweeping an ultrasonic probe through a plurality of rib interspaces while acquiring three-dimensional ultrasonic image data. In a seventh example of the method, optionally including one or more or each of the first example to the sixth example, the ultrasonic probe is swept through a plurality of rib interspaces in a longitudinal dividing motion from the bottom of the thoracic cavity to the top of the thoracic cavity, and vice versa.

[0081] The present disclosure also provides support for a method for ultrasonic imaging, the method comprising: acquiring dynamic three-dimensional ultrasonic image data; identifying a plurality of rib spaces in the dynamic three-dimensional ultrasonic image data; identifying and exposing the pleural surfaces in each of the plurality of rib spaces; using the image characteristics of the pleural surfaces to sequentially align the plurality of rib spaces according to anatomical order; synchronizing the display timing of each of the plurality of rib spaces such that the display start time and the display end time of each rib space occur simultaneously; and displaying the dynamic three-dimensional ultrasonic image data as a dynamic three-dimensional ultrasonic image. In a first example of the method, sequentially aligning the plurality of rib spaces includes: aligning the plurality of rib spaces based on the characteristics of each of the plurality of rib spaces, the characteristics including the characteristics of the pleural surfaces. In a second example of the method, which optionally includes the first example, synchronizing the display timing includes: adjusting the display playback speed of one or more rib spaces. In a third example of the method, which optionally includes one or both of the first example and the second example, synchronizing the display timing includes: identifying the respiratory cycle in each of the plurality of rib spaces; and synchronizing the respiratory cycles of the plurality of rib spaces. In a fourth example of the method, which optionally includes one or more or each of the first example to the third example, identifying the plurality of rib spaces in the dynamic three-dimensional ultrasonic image data includes: detecting a plurality of rib shadows in the dynamic three-dimensional ultrasonic image data; and using the plurality of rib shadows as boundaries between each of the plurality of rib spaces. In a fifth example of the method, which optionally includes one or more or each of the first example to the fourth example, using the plurality of rib shadows as boundaries includes: segmenting the dynamic three-dimensional ultrasonic image data into a plurality of volumes, the plurality of volumes showing portions of the dynamic three-dimensional ultrasonic image data, each portion including a single rib space. In a sixth example of the method, which optionally includes one or more or each of the first example to the fifth example, sequentially aligning the plurality of rib spaces includes: arranging the plurality of rib spaces according to the anatomical order based on the image characteristics of each rib space.

[0082] The present disclosure also provides support for an ultrasound imaging system that includes: a display device; an ultrasound probe configured to acquire dynamic three-dimensional ultrasound image data; and a processor in electronic communication with the ultrasound probe and the display device, wherein the processor is configured with computer-readable instructions stored on a non-transitory memory, the computer-readable instructions, when executed, cause the processor to perform the following operations: generate a plurality of three-dimensional rib interspace segments from the dynamic three-dimensional ultrasound image data; generate a dynamic panoramic three-dimensional ultrasound image depicting the plurality of three-dimensional rib interspace segments in anatomical order; and display the dynamic panoramic three-dimensional ultrasound image on the display device, wherein the amount of play time for the plurality of three-dimensional rib interspace segments is the same. In a first example of the system, the system further includes: an ultrasound configured to capture dynamic three-dimensional ultrasound image data as the ultrasound probe sweeps through an imaging subject. In a second example of the system that optionally includes the first example, the ultrasound probe is a matrix array probe. In a third example of the system that optionally includes one or both of the first example and the second example, the processor is further configured with instructions in the non-transitory memory that, when executed, cause the processor to perform the following operations: generate the plurality of three-dimensional rib interspace segments in real time as the ultrasound probe sweeps through the corresponding rib interspaces; and identify and expose the pleural surface of each of the plurality of three-dimensional rib interspace segments. In a fourth example of the system that optionally includes one or more or each of the first example to the third example, the processor is further configured with instructions in the non-transitory memory that, when executed, cause the processor to perform the following operations: synchronize the display timing of each of the plurality of three-dimensional rib interspace segments in time such that the display start time and end time for each rib interspace occur simultaneously.

[0083] As used herein, an element or step recited in the singular and preceded by the word "a" or "an" should be understood as not excluding a plurality of the recited elements or steps, unless expressly stated to the contrary. Further, a reference to "one embodiment" of the present invention is not to be construed as excluding the existence of additional embodiments that also incorporate the recited features. Additionally, unless expressly stated to the contrary, an embodiment that "includes," "comprises," or "has" an element or elements with a particular characteristic may include additional such elements that do not have that particular characteristic. The terms "includes" and "in" are used as plain language equivalents of the corresponding terms "comprises" and "wherein." Further, the terms "first," "second," and "third," etc. are used merely as labels and are not intended to impose numerical requirements or a particular positional order on their objects.

[0084] The embodiments of the present disclosure shown in the accompanying drawings and described above are only exemplary embodiments and are not intended to limit the scope of the appended claims, including any equivalents included within the scope of the claims. Various modifications are possible and will be apparent to those skilled in the art. Any combination of the non-mutually exclusive features described herein is intended to be within the scope of the invention. That is, the features of the embodiments may be combined with any suitable aspect described above, and the optional features of any one aspect may be combined with any other suitable aspect. Similarly, the features listed in the dependent claims may be combined with the non-mutually exclusive features of other dependent claims, particularly where the dependent claims depend on the same independent claim. Since some jurisdictions require single claim dependencies, these dependencies may have been used in practice, but this should not be taken to mean that the features in the dependent claims are mutually exclusive.

Claims

1. A method for ultrasonic imaging, the method comprising: Acquiring dynamic three-dimensional ultrasonic image data; Generating a plurality of dynamic three-dimensional rib space segments from the dynamic three-dimensional ultrasonic image data; Generating a dynamic panoramic three-dimensional ultrasonic image depicting the plurality of dynamic three-dimensional rib space segments in anatomical order; Synchronizing the plurality of dynamic three-dimensional rib space segments in time; And Outputting the dynamic panoramic three-dimensional ultrasonic image for display.

2. The method according to claim 1, wherein generating the plurality of dynamic three-dimensional rib space segments comprises: Automatically identifying a plurality of rib shadows in the dynamic three-dimensional ultrasonic image data; Using the plurality of rib shadows to define boundaries between each of the plurality of rib spaces; And Based on the boundaries between each of the plurality of rib spaces, segmenting the dynamic three-dimensional ultrasonic image data into a plurality of volumes.

3. The method according to claim 2, wherein the plurality of rib shadows are detected by tracking the movement of an ultrasonic probe used to capture the dynamic three-dimensional ultrasonic image data during an imaging scan.

4. The method according to claim 2, wherein the plurality of rib shadows are detected according to the intensity and / or brightness of pixels of the dynamic three-dimensional ultrasonic image data.

5. The method according to claim 1, wherein depicting the plurality of dynamic three-dimensional rib interspace segments in the anatomical order comprises: Identifying and exposing the pleural surface of each three-dimensional rib space segment; And sorting the plurality of dynamic three-dimensional rib space segments according to the anatomical order based on the pleural surface.

6. The method according to claim 1, wherein synchronizing the plurality of dynamic three-dimensional rib gap segments in time comprises: Applying time scaling to expand and / or contract one or more of the plurality of dynamic three-dimensional rib space segments.

7. The method according to claim 1, wherein the dynamic three-dimensional ultrasonic image data is acquired by sweeping an ultrasonic probe through a plurality of rib spaces while acquiring three-dimensional ultrasonic image data.

8. The method according to claim 7, wherein the ultrasonic probe is swept through a plurality of rib spaces in a longitudinal dividing motion from the bottom of the chest cavity to the top of the chest cavity and vice versa.

9. A method for ultrasonic imaging, the method comprising: Acquiring dynamic three-dimensional ultrasonic image data; Identifying a plurality of rib spaces in the dynamic three-dimensional ultrasonic image data; Identifying and exposing the pleural surface in each of the plurality of rib spaces; Using the image characteristics of the pleural surface to sequentially align the plurality of rib spaces according to anatomical order; Synchronizing the display timing of each of the plurality of rib spaces such that the display start time and end time of each rib space occur simultaneously; and Displaying the dynamic three-dimensional ultrasonic image data as a dynamic three-dimensional ultrasonic image.

10. The method according to claim 9, wherein sequentially aligning the plurality of rib spaces comprises: Aligning the plurality of rib spaces based on the characteristics of each of the plurality of rib spaces, the characteristics including the characteristics of the pleural surface.

11. The method according to claim 9, wherein synchronizing the display timing comprises adjusting the display playback speed of one or more rib spaces.

12. The method according to claim 9, wherein synchronizing the display timing includes identifying a respiratory cycle in each of the plurality of rib spaces; and synchronizing the respiratory cycles of the plurality of rib spaces.

13. The method according to claim 9, wherein identifying the plurality of intercostal spaces in the dynamic three-dimensional ultrasound image data comprises: Detecting a plurality of rib shadows in the dynamic three-dimensional ultrasound image data; And using the plurality of rib shadows as boundaries between each of the plurality of rib spaces.

14. The method according to claim 13, wherein using the plurality of rib shadows as boundaries includes segmenting the dynamic three-dimensional ultrasound image data into a plurality of volumes, the plurality of volumes showing portions of the dynamic three-dimensional ultrasound image data, each portion including a single rib space.

15. The method according to claim 9, wherein sequentially aligning the plurality of rib spaces includes arranging the plurality of rib spaces according to the anatomical order based on the image characteristics of each rib space.

Images