Method and system for generating 3D pleural surface

Abstract

The invention relates to a method and system for generating a 3D pleural surface. Various methods and systems for a medical imaging system are provided. In one embodiment, a method (300, 1100) includes generating a three-dimensional (3D) pleural surface as viewed from inside and outward of the lung based on ultrasound imaging signals. The 3D pleural surface can be generated from 2D or 3D ultrasound images. The method further includes shading the generated 3D pleural surface via at least one virtual light source positioned as inside the lung.

Description

Technical Field

[0001] Embodiments of the subject matter disclosed herein relate to ultrasound imaging, and in particular to visualizing the lung pleura as a three-dimensional (3D) surface viewed from the inside of the lung and looking outward. Background Art

[0002] Ultrasound imaging systems typically include an ultrasound probe applied to a patient's body and a workstation or device operably coupled to the probe. During scanning, the probe can be controlled by an operator of the system and is configured to transmit and receive ultrasound signals that are processed by the workstation or device into ultrasound images. The workstation or device can display the ultrasound image and multiple user-selectable inputs through a display device. The operator or other user can interact with the workstation or device to analyze the image displayed on and / or selected from the multiple user-selectable inputs.

[0003] As an example, ultrasound imaging can be used to examine a patient's lungs due to, for example, their ease of use at the point of care and resource availability relative to chest x-rays or chest computed tomography (CT) scans. In addition, ultrasound imaging systems do not expose the patient to radiation. Lung ultrasound imaging, also known as lung sonography, includes interpreting the topography of the lung pleura for diagnostic purposes. Summary of the invention

[0004] This summary introduces concepts that are described in more detail in the detailed description. It should not be used to identify essential features of the claimed subject matter, nor should it be used to limit the scope of the claimed subject matter.

[0005] In one aspect, a method for imaging a patient's lungs includes generating a three-dimensional (3D) pleural surface viewed from the inside of the lungs and looking outward based on ultrasound imaging signals. The 3D pleural surface can be generated from a 2D or 3D ultrasound image. The method also includes shading the generated 3D pleural surface via at least one virtual light source positioned as if on the inside of the lungs.

[0006] It should be understood that the above brief description is provided to introduce in a simplified form selected concepts that are further described in the detailed description. It is not meant to identify key or essential features of the claimed subject matter, the scope of which is solely defined by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to implementations that solve any disadvantages noted above or in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The present disclosure will be better understood by reading the following description of non-limiting embodiments with reference to the accompanying drawings, in which:

[0008] Figure 1shows a block schematic diagram of an ultrasound imaging system according to an embodiment;

[0009] Figure 2 is a schematic diagram illustrating an image processing system for generating a 3D pleural surface according to an embodiment;

[0010] Figure 3 A flow chart illustrating an example method for generating a 3D pleural surface from a 2D ultrasound image according to an embodiment is shown;

[0011] Figure 4 shows a first example annotated 2D lung ultrasound image that can be used to generate a 3D pleural surface according to an embodiment;

[0012] Figure 5 An example method for generating a 3D pleural surface by stacking 2D ultrasound images according to an embodiment is shown;

[0013] Figure 6 An example method for generating a 3D pleural surface by generating a 3D mesh from a 2D ultrasound image according to an embodiment is shown;

[0014] Figure 7 shows a flow chart illustrating an example method for shading a 3D pleural surface according to an embodiment;

[0015] Figure 8 showing a first example display including a 3D pleural surface according to an embodiment;

[0016] Fig. 9 showing a second example display including a 3D pleural surface according to an embodiment;

[0017] Fig.10 showing a third example display including a 3D pleural surface according to an embodiment;

[0018] Fig.11 A flow chart illustrating an example method for generating a 3D pleural surface from a 3D ultrasound image according to an embodiment is shown;

[0019] Fig.12 Shown according to Fig.11 A set of example images generated by the method; and

[0020] Fig.13 A fourth example display is shown including a bi-plane and 3D display of the lungs. DETAILED DESCRIPTION

[0021] Now refer to Figures 1 to 13Embodiments of the present disclosure are described by way of example, and they relate to various embodiments for generating a 3D representation of the pleural surface of the lung pleura viewed from the inside of the lung and looking outward. It should be understood that, as described herein, a "3D pleural surface" is a 3D computer rendering representing an anatomical region (e.g., pleura) captured using an imaging system. In an inflated lung, the pleura that forms the outer boundary of the lung against the chest wall can provide essentially the only anatomical lung structure that can be detected by ultrasound. The pleura is presented in a 2D ultrasound image as a high-echo horizontal segment of brighter (e.g., whiter) pixels, called the pleural line, which moves synchronously with breathing in a phenomenon called pleural sliding. The 3D pleural surface includes the volumetric morphology of the lung pleura. Some morphologies of the pleura are shown in a 2D ultrasound image, but further details of the pleural morphology can be visualized in the 3D pleural surface, which can help identify and / or diagnose lung conditions that can be visualized as irregularities in the pleural surface morphology. The 3D pleural surface can show the pleural topography in a wider area of ​​the lung than can be captured in a 2D ultrasound image. For example, the 3D pleural surface can be a live 3D surface rendering of the pleura, or can be a static image provided after data collection using an ultrasound probe. In some embodiments, the 3D pleural surface described herein can be a 4D image, wherein the 3D pleural surface is a 3D image that can change over time (e.g., in real time) to reflect changes in the patient's anatomy visualized using the ultrasound probe (e.g., due to patient breathing, movement, etc.).

[0022] Based on the imaging system (such as Figure 1 The 3D pleural surface is generated by using medical imaging data acquired by an ultrasound imaging system (such as the one shown in FIG. 1 ). Since the processes described herein can be applied to preprocessed imaging data and / or processed images, the term "image" is generally used throughout this disclosure to refer to preprocessed and partially processed image data (e.g., pre-beamformed RF or I / Q data, pre-scan converted RF data) as well as fully processed images (e.g., scan converted and filtered images ready for display). Figure 2 An example image processing system that can be used to generate a 3D pleural surface is shown in FIG. The image processing system can use image processing techniques and one or more algorithms to identify the pleural line in ultrasound imaging data and generate a 3D pleural surface viewed from the inside of the lung and looking outward. Figure 3 As described in the method of , a 3D pleural surface can be generated from a 2D ultrasound image. Various visual markers or indicators can be used to identify the pleural line in the 2D ultrasound image, such as Figure 4 In some embodiments, as described in Figure 5 As described, generating a 3D pleural surface includes stacking multiple 2D ultrasound images of the same lung in which the pleural line has been identified. In some embodiments, such as with respect to Figure 6As described, a 3D pleural surface is generated from a mesh based on a series of 2D ultrasound images. In some embodiments, as described with respect to Fig.11 The method described above generates 3D pleural surfaces from 3D volume images. Fig.12 Shown according to Fig.11 Example 3D ultrasound images generated by the method. Figure 7 As described in the method, after generating the 3D pleural surface, shading can be applied to highlight irregularities of the 3D pleural surface. Figures 8 to 10 An example display including a 3D pleural surface viewed from the inside of a lung and looking outward is shown. Fig.13 Biplane and 3D images of the lungs are shown. In this way, pleural irregularities can be highlighted in a display of the 3D pleural surface generated in real time from volumetric ultrasound imaging data, thereby reducing the time until a diagnosis can be made and reducing both intra-operator and inter-operator variability.

[0023] An advantage that can be achieved in the practice of some embodiments of the described systems and techniques is that inconsistencies in the detection of pleural irregularities can be reduced, particularly between different operators. This may be particularly advantageous for improving the detection accuracy of point-of-care ultrasound operators who may have less training than ultrasound experts (e.g., sonographers or radiologists). For example, an emergency room physician who may not have received expert-level ultrasound training may be more likely to overlook irregularities or incorrectly identify normal structures or imaging artifacts as irregularities, which may increase the burden of follow-up scans for the radiology department and increase patient discomfort. In addition, by reducing the mental burden of follow-up scans and point-of-care ultrasound operators, the amount of time it takes to make an accurate diagnosis can be reduced. Although the systems and methods for evaluating medical images described below are discussed with reference to ultrasound imaging systems, it can be noted that the methods described herein can be applied to a variety of imaging systems (e.g., MRI, PET, x-ray, CT, or other similar systems).

[0024] refer to Figure 1, a schematic diagram of an ultrasound imaging system 100 according to an embodiment of the present disclosure is shown. However, it is understood that the embodiments described herein can be implemented using other types of medical imaging modalities (e.g., magnetic resonance imaging, computed tomography, positron emission tomography, etc.). The ultrasound imaging system 100 includes a transmit beamformer 101 and a transmitter 102, which drives the transducer elements 104 within the transducer array (referred to herein as the probe 106) to transmit pulsed ultrasound signals (referred to herein as transmit pulses) into the body (not shown). According to an embodiment, the probe 106 can be a one-dimensional (1D) transducer array probe. In some embodiments, the probe 106 can be a two-dimensional (2D) matrix transducer array probe. In other embodiments, the probe 106 can be a 1.5-dimensional (1.5D) probe or any ultrasound probe capable of live 3D imaging, such as a matrix array probe. The transducer element 104 can be made of a piezoelectric material. When a voltage is applied to the piezoelectric material, the piezoelectric material physically expands and contracts, thereby emitting an ultrasound ball wave. In this way, the transducer element 104 may convert the electronic transmit signal into an acoustic transmit beam.

[0025] After the transducer elements 104 of the probe 106 transmit pulsed ultrasound signals into the body (of the patient), the pulsed ultrasound signals are backscattered from structures inside the body (such as blood cells or muscle tissue) to produce echoes that return to the transducer elements 104. These echoes are converted into electrical signals or ultrasound data by the transducer elements 104, and the electrical signals are received by the receiver 108. The electrical signals representing the received echoes pass through a receive beamformer 110 that performs beamforming and outputs ultrasound data, which may be in the form of a radio frequency (RF) signal. In addition, the transducer elements 104 may generate one or more ultrasound pulses based on the received echoes to form one or more transmit beams.

[0026] According to some embodiments, the probe 106 may contain electronic circuitry to perform all or part of transmit beamforming and / or receive beamforming. For example, all or part of the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 may be positioned within the probe 106. In the present disclosure, the term "scan" or "scanning" may also be used to refer to the acquisition of data by the process of transmitting and receiving ultrasound signals. In the present disclosure, the term "data" may be used to refer to one or more data sets acquired using an ultrasound imaging system.

[0027] The user interface 115 may be used to control the operation of the ultrasound imaging system 100, including controlling the input of patient data (e.g., patient medical history), changing scanning or display parameters, initiating a probe repolarization sequence, etc. The user interface 115 may include one or more of a rotary element, a mouse, a keyboard, a trackball, hard keys linked to specific actions, soft keys that may be configured to control different functions, and a graphical user interface displayed on the display device 118. In some embodiments, the display device 118 may include a touch-sensitive display, and thus the display device 118 may be included in the user interface 115.

[0028] The ultrasound imaging system 100 also includes a processor 116, which is used to control the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110. The processor 116 is in electronic communication (e.g., communicatively connected) with the probe 106. As used herein, the term "electronic communication" may be defined as including both wired communication and wireless communication. The processor 116 may control the probe 106 to acquire data according to instructions stored in the processor's memory and / or the memory 120. As an example, the processor 116 controls which transducer elements of the transducer elements 104 are active and the shape of the beam emitted from the probe 106. The processor 116 also communicates electronically with the display device 118, and the processor 116 may process data (e.g., ultrasound data) into an image for display on the display device 118. According to one embodiment, the processor 116 may include a central processing unit (CPU). According to other embodiments, the processor 116 may include other electronic components capable of performing processing functions, such as a digital signal processor, a field programmable gate array (FPGA), or a graphics board. According to other embodiments, the processor 116 may include a plurality of electronic components capable of performing processing functions. For example, the processor 116 may include two or more electronic components selected from a list of electronic components including: a central processing unit, a digital signal processor, a field programmable gate array, and a graphics board. According to another embodiment, the processor 116 may also include a composite demodulator (not shown) that demodulates the RF data and generates raw data. In another embodiment, demodulation may be performed earlier in the processing chain.

[0029] The processor 116 is adapted to perform one or more processing operations according to a plurality of optional ultrasound modalities on the data. In one example, the data may be processed in real time during a scanning session because the echo signals are received by the receiver 108 and transmitted to the processor 116. For the purposes of this disclosure, the term "real time" is defined as including a process that is performed without any intentional delay (e.g., substantially when it occurs). For example, an embodiment may acquire images at a real-time rate of 7 frames / second to 20 frames / second. The ultrasound imaging system 100 is capable of acquiring two-dimensional (2D) data of one or more planes at a significantly faster rate. However, it should be understood that the real-time frame rate may depend on the length of time (e.g., duration) spent to acquire and / or process each frame of data for display. Therefore, when a relatively large amount of data is acquired, the real-time frame rate may be slower. For example, the ultrasound imaging system 100 may additionally or alternatively acquire three-dimensional (3D) data. Therefore, some embodiments may have a real-time frame rate significantly faster than 20 frames / second, while other embodiments may have a real-time frame rate lower than 7 frames / second.

[0030] In some embodiments, data may be temporarily stored in a buffer (not shown) during a scanning session and processed in less than real time in live or offline (e.g., frozen) operation. Some embodiments of the present disclosure may include multiple processors (not shown) to handle processing tasks handled by processor 116 according to the exemplary embodiments described above. For example, a first processor may be utilized to demodulate and extract the RF signal before displaying an image, while a second processor may be utilized to further process the data (e.g., by augmenting the data as further described herein). It should be appreciated that other embodiments may use different processor arrangements.

[0031] The ultrasound imaging system 100 can continuously acquire data at a frame rate of, for example, 10 Hz to 30 Hz (e.g., 10 to 30 frames per second). An image generated based on the data can be refreshed at a similar frame rate on the display device 118. Other embodiments can acquire and display data at different rates. For example, depending on the size of the frame and the intended application, some embodiments can acquire data at a frame rate less than 10 Hz or greater than 30 Hz. The memory 120 can store processed frames of acquired data. In an exemplary embodiment, the memory 120 has sufficient capacity to store at least a few seconds of ultrasound data frames. The data frames are stored in a manner that is easy to retrieve according to their acquisition order or time. The memory 120 may include any known data storage medium.

[0032] In various embodiments of the present disclosure, the data may be processed by the processor 116 in different mode-related modules to form a 2D or 3D image. When multiple images are obtained, the processor 116 may also be configured to stabilize or register the images. For example, one or more modules may generate B-mode, color Doppler, M-mode, color M-mode, color flow imaging, spectral Doppler, elastic imaging, tissue velocity imaging (TVI), strain, strain rate, etc., and combinations thereof. As an example, one or more modules may process color Doppler data, which may include traditional color flow Doppler, power Doppler, high-definition (HD) flow Doppler, etc. The image lines and / or frames are stored in a memory, and may include timing information indicating the time when the image lines and / or frames are stored in the memory. These modules may include, for example, a scan conversion module that performs a scan conversion operation to convert the acquired image from beam space coordinates to display space coordinates. A video processor module may be provided that reads the acquired image from the memory and displays the image in real time when a procedure (e.g., ultrasound imaging) is performed on the patient. The video processor module may include a separate image memory, and the ultrasound image may be written to the image memory for reading and display by the display device 118 .

[0033] In addition, the components of the ultrasound imaging system 100 may be coupled to each other to form a single structure, may be separate but located in a common room, or may be remote relative to each other. For example, one or more of the modules described herein may operate in a data server that has a different and remote location relative to other components of the ultrasound imaging system 100, such as the probe 106 and the user interface 115. Optionally, the ultrasound imaging system 100 may be a single system that can be moved from one room (e.g., portable) to another room. For example, the ultrasound imaging system 100 may include wheels or may be transportable on a cart, or may include a handheld device.

[0034] For example, in various embodiments of the present disclosure, one or more components of the ultrasound imaging system 100 may be included in a portable handheld ultrasound imaging device. For example, the display device 118 and the user interface 115 may be integrated into the external surface of the handheld ultrasound imaging device, which may also include a processor 116 and a memory 120 therein. The probe 106 may include a handheld probe that electronically communicates with the handheld ultrasound imaging device to collect raw ultrasound data. The transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 may be included in the same or different parts of the ultrasound imaging system 100. For example, the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 may be included in a handheld ultrasound imaging device, a probe, and a combination thereof.

[0035] See also Figure 2 , shows an exemplary medical image processing system 200. In some embodiments, the medical image processing system 200 is incorporated into a medical imaging system such as an ultrasound imaging system (e.g., Figure 1 In some embodiments, at least a portion of the medical image processing system 200 is disposed at a device (e.g., an edge device or a server) that is communicatively coupled to the medical imaging system via a wired and / or wireless connection. In some embodiments, the medical image processing system 200 is disposed at a separate device (e.g., a workstation) that can receive images from the medical imaging system or from a storage device that stores images generated by the medical imaging system. The medical image processing system 200 may include an image processor 231, a user input device 232, and a display device 233. For example, the image processor 231 may be operably / communicatively coupled to the user input device 232 and the display device 233.

[0036] The image processor 231 includes a processor 204 configured to execute machine-readable instructions stored in a non-transitory memory 206. The processor 204 may be single-core or multi-core, and the program executed by the processor 204 may be configured for parallel or distributed processing. In some embodiments, the processor 204 may optionally include separate components distributed throughout two or more devices, which may be remotely located and / or configured for collaborative processing. In some embodiments, one or more aspects of the processor 204 may be virtualized and performed by a remotely accessible networked computing device configured in a cloud computing configuration. In some embodiments, the processor 204 may include other electronic components capable of performing processing functions, such as a digital signal processor, a field programmable gate array (FPGA), or a graphics board. In some embodiments, the processor 204 may include multiple electronic components capable of performing processing functions. For example, the processor 204 may include two or more electronic components selected from multiple possible electronic components including the following items: a central processing unit, a digital signal processor, a field programmable gate array, and a graphics board. In yet other embodiments, processor 204 may be configured as a graphics processing unit (GPU), including a parallel computing architecture and parallel processing capabilities.

[0037] exist Figure 2 In the illustrated embodiment, the non-transitory memory 206 stores a 3D generation module 212 and medical image data 214. The 3D generation module 212 includes one or more algorithms for processing an input medical image from the medical image data 214. Figure 3 and Fig.11As described, the 3D generation module 212 may generate a 3D pleural surface from a 2D and / or 3D ultrasound image. For example, the 3D generation module 212 may include one or more image recognition algorithms, shape or edge detection algorithms, gradient algorithms, etc., for processing input medical images. Additionally or alternatively, the 3D generation module 212 may store instructions for implementing a neural network (such as a convolutional neural network) to detect the pleural surface / pleura captured in the medical image data 214 in real time. For example, the 3D generation module 212 may include a trained and / or untrained neural network and may also include a training routine or parameters (e.g., weights and biases) associated with one or more neural network models stored therein. In some embodiments, the 3D generation module 212 may evaluate the medical image data 214 when the medical image data 214 is acquired in real time. Additionally or alternatively, the 3D generation module 212 may evaluate the medical image data 214 offline rather than in real time.

[0038] For example, when the medical image data 214 includes lung ultrasound data, the identified anatomical features may include the lung pleura, which may be identified by the 3D generation module 212 based on pleural sliding via edge detection techniques and / or gradient changes. Figure 3 and Fig.11 As described in detail, detection of pleural location can help indicate areas of an ultrasound image that are noise and can be removed to expose the pleural surface.

[0039] Optionally, the image processor 231 is communicatively coupled to a training module 210 that includes instructions for training one or more of the machine learning models stored in the 3D generation module 212. The training module 210 may include instructions that, when executed by the processor, cause the processor to build a model (e.g., a mathematical model) based on sample data to make predictions or decisions about the detection and classification of anatomical irregularities without explicit programming of traditional algorithms that do not utilize machine learning. In one example, the training module 210 includes instructions for receiving a training data set from medical image data 214. The training data set includes a set of medical images, associated ground truth labels / images, and associated model outputs for training one or more of the machine learning models stored in the 3D generation module 212. The training module 210 may receive medical images, associated ground truth labels / images, and associated model outputs for training one or more machine learning models from sources other than the medical image data 214 (such as other image processing systems, the cloud, etc.). In some embodiments, one or more aspects of the training module 210 may include a remotely accessible networked storage device configured in a cloud computing configuration. Furthermore, in some embodiments, the training module 210 is included in the non-transitory memory 206. Additionally or alternatively, in some embodiments, the training module 210 can be used to generate the 3D generation module 212 offline and remote from the image processing system 200. In such embodiments, the training module 210 may not be included in the image processing system 200, but may generate data stored in the image processing system 200. For example, the 3D generation module 212 may be pre-trained using the training module 210 at a manufacturing site.

[0040] The non-transitory memory 206 also stores medical image data 214. The medical image data 214 includes, for example, functional images and / or anatomical images captured by an imaging modality such as an ultrasound imaging system, an MRI system, a CT system, a PET system, etc. As an example, the medical image data 214 may include an ultrasound image, such as a lung ultrasound image. In addition, the medical image data 214 may include one or more of a 2D image, a 3D image, a static single-frame image, and a multi-frame image loop (e.g., a movie).

[0041] In some embodiments, non-transitory memory 206 may include components disposed at two or more devices that may be remotely located and / or configured for collaborative processing. In some embodiments, one or more aspects of non-transitory memory 206 may include a remotely accessible networked storage device configured with cloud computing. As an example, non-transitory memory 206 may be part of a picture archiving and communication system (PACS) that is configured to store, for example, patient medical history, imaging data, test results, diagnostic information, management information, and / or scheduling information.

[0042] Image processing system 200 may also include user input device 232. User input device 232 may include one or more of a touch screen, keyboard, mouse, trackpad, motion sensing camera, or other device configured to enable a user to interact with and manipulate data stored within image processor 231.

[0043] The display device 233 may include one or more display devices utilizing any type of display technology. In some embodiments, the display device 233 may include a computer monitor and may display unprocessed images, processed images, parameter mapping maps, and / or inspection reports. The display device 233 may be combined with the processor 204, the non-transient memory 206, and / or the user input device 232 in a shared housing, or may be a peripheral display device. The display device 233 may include a monitor, a touch screen, a projector, or another type of display device that enables a user to view medical images and / or interact with various data stored in the non-transient memory 206. In some embodiments, the display device 233 may be included in a smart phone, a tablet computer, a smart watch, etc.

[0044] Understandably, Figure 2 The illustrated medical image processing system 200 is one non-limiting embodiment of an image processing system, and other imaging processing systems may include more, fewer, or different components without departing from the scope of the present disclosure. In addition, in some embodiments, at least a portion of the medical image processing system 200 may include Figure 1 The ultrasound imaging system 100 may be included in the medical image processing system 200, or vice versa (eg, at least a portion of the ultrasound imaging system 100 may be included in the medical image processing system 200).

[0045] As used herein, the terms "system" and "module" may include hardware and / or software systems that operate to perform one or more functions. For example, a module or system may include, or be included in, a computer processor, a controller, or other logic-based device that performs operations based on instructions stored on a tangible and non-transitory computer-readable storage medium (such as a computer memory). Alternatively, a module or system may include a hard-wired device that performs operations based on the hard-wired logic of the device. The various modules or systems shown in the figures may represent hardware that operates based on software or hard-wired instructions, software that instructs hardware to perform operations, or a combination thereof.

[0046] A "system" or "module" may include or represent hardware and related instructions (e.g., software stored on a tangible and non-transitory computer-readable storage medium (such as a computer hard drive, ROM, RAM, etc.) that perform one or more operations described herein. The hardware may include electronic circuitry that includes and / or is connected to one or more logic-based devices, such as microprocessors, processors, controllers, etc. These devices may be off-the-shelf devices that are appropriately programmed or instructed to perform the operations described herein in accordance with the instructions described above. Additionally or alternatively, one or more of these devices may be hardwired with logic circuitry to perform these operations.

[0047] Figure 3 A flow chart of an example method 300 for generating a 3D pleural surface is shown, wherein ultrasound imaging signals for generating a 3D pleural surface are acquired by positioning a 2D probe to acquire volumetric ultrasound data. The 3D pleural surface is viewed from the inside of the lung and looking outward. Specifically, method 300 provides a workflow for generating a 3D pleural surface from a series of 2D images by processing a series of 2D images captured using a 2D probe to generate a stack of 2D images, and / or by generating a 3D mesh from the series of 2D images (from which a 3D image can be generated). The method 300 is directed to an example method for generating a 3D pleural surface using an ultrasound imaging system such as a 2D probe. Figure 1 Method 300 is described using a 2D ultrasound image acquired by an ultrasound imaging system 100 of the type described above, but other ultrasound imaging systems may also be used. In addition, method 300 may be applicable to other imaging modalities. Method 300 may be implemented by one or more of the above-described systems, including Figure 1 The ultrasound imaging system 100 and Figure 2 Therefore, the method 300 may be stored as executable instructions in a non-transitory memory (such as Figure 1 The memory 120 and / or Figure 2 206) and by a processor such as Figure 1 The processor 116 and / or Figure 2In some embodiments, method 300 is performed in real time while the 2D images are being acquired, while in other embodiments, at least a portion of method 300 is performed offline after the 2D images are acquired. For example, the processor may evaluate the 2D images stored in the memory even when the ultrasound system is not actively operating to acquire images (when data acquisition is frozen). In addition, at least a portion of method 300 may be performed in parallel. For example, ultrasound data for a second 2D image may be acquired while the first 2D image is being generated, ultrasound data for a third 2D image may be acquired while the first 2D image is being analyzed, and so on.

[0048] At 302, method 300 includes receiving a lung ultrasound protocol selection. The lung ultrasound protocol may be selected by an operator (e.g., a user) of the ultrasound imaging system through a user interface (e.g., user interface 115). As an example, the operator may select a lung ultrasound protocol from a plurality of possible ultrasound protocols using a drop-down menu or by selecting a virtual button. Alternatively, the system may automatically select a protocol based on data received from an electronic health record (EHR) associated with the patient. For example, the EHR may include previously performed examinations, diagnoses, and current treatments, which may be used to select a lung ultrasound protocol. In addition, in some examples, the operator may manually input and / or update parameters for the lung ultrasound protocol. The lung ultrasound protocol may be a system-guided protocol in which the system guides the operator step by step through the protocol, or may be a user-guided protocol in which the operator follows a laboratory-defined or customized protocol, and the system does not enforce a specific protocol or does not know the protocol steps in advance.

[0049] In addition, the lung ultrasound protocol may include multiple scanning locations (e.g., views), probe movements, and / or imaging modes that are performed sequentially. For example, the lung ultrasound protocol may include the use of real-time B-mode imaging that utilizes a convex, curved, or linear ultrasound probe (e.g., Figure 1 The lung ultrasound protocol may also include the use of a dynamic M-mode. The lung ultrasound protocol may include a longitudinal scan, in which the probe is positioned perpendicular to the ribs, and / or an oblique scan, in which the probe is positioned along the intercostal spaces between the ribs. Additionally, in some examples, the lung ultrasound protocol may include a panoramic scan, in which the user waves the ultrasound probe downward from the head, and multiple views from the scan may be stitched together to provide anatomical and spatial relationships.

[0050] At 304, method 300 includes collecting ultrasound data with an ultrasound probe by transmitting and receiving ultrasound signals according to a lung ultrasound protocol. Collecting ultrasound data according to the lung ultrasound protocol may include the system displaying instructions on a user interface, for example, to guide an operator to complete the collection of a specified scanning site. Additionally or alternatively, the lung ultrasound protocol may include instructions for the ultrasound system to automatically collect some or all data or perform other functions. For example, the lung ultrasound protocol may include instructions for the user to move, rotate, tilt and / or wave the ultrasound probe and automatically start and / or terminate the scanning process and / or adjust the imaging parameters of the ultrasound probe, such as ultrasound signal transmission parameters, ultrasound signal reception parameters, ultrasound signal processing parameters, or ultrasound signal display parameters. In addition, the collected ultrasound data includes one or more image parameters calculated for each pixel or a group of pixels to be displayed (e.g., a group of pixels assigned the same parameter value), wherein the one or more calculated image parameters include, for example, one or more of intensity, velocity, color blood flow velocity, texture, granularity, contractility, deformation, and deformation rate values.

[0051] At 306, method 300 includes generating an ultrasound image from the acquired ultrasound data. For example, the signal data acquired at 304 during the method is processed and analyzed by a processor to generate a 2D ultrasound image at a specified frame rate. The processor may include an image processing module that receives the signal data (e.g., image data) acquired at 304 and processes the received image data. For example, the image processing module may process the ultrasound signal to generate a slice or frame of ultrasound information (e.g., a 2D ultrasound image) for display to an operator. In one example, generating a 2D image may include determining an intensity value for each pixel to be displayed based on the received image data. The 2D ultrasound image will also be referred to herein as a "frame" or "image frame."

[0052] At 308, method 300 includes detecting a bottom edge of the pleura in each 2D ultrasound image. The pleura appears in the 2D ultrasound image as a high echogenic horizontal segment of brighter (e.g., whiter) pixels, called the pleural line, which moves synchronously with respiration in a phenomenon called pleural sliding. Detecting the bottom edge of the pleura may include identifying a lower boundary and an upper boundary of the pleura based on brightness changes between pixels, such as by using edge detection techniques or gradient changes. For example, the processor may apply a method such as included in Figure 2An edge detection algorithm in the 3D generation module 212 includes one or more mathematical methods for identifying points (e.g., pixels) where the image brightness changes sharply and / or has a discontinuity to identify the lower and upper boundaries of the pleural line. As an example, the processor may apply the edge detection algorithm to the area with the highest local variation. For another example, additionally or alternatively, the gradient algorithm may identify the local maximum and / or minimum pixel brightness at the pleural position to identify the lower and upper boundaries of the pleural line in each image. In some embodiments, a deep learning model trained for real-time or static (e.g., non-real-time) pleural detection is used to identify the bottom edge of the pleura.

[0053] The lower and upper boundaries of the pleura may include subpleural consolidation. For example, the bottom edge (e.g., as further described herein, indicating a boundary below which ultrasound data is removed) may be positioned at a predetermined distance from the pleural line toward the lung. It will be appreciated that, conventionally, the subpleural consolidation extends a maximum distance 'n' from the pleural surface toward the lung. Thus, the bottom edge of the pleura may be positioned a distance 'n' from the lower boundary of the pleura identified as described above. In other embodiments, the same or different edge detection algorithms as described above may be used to identify subpleural consolidation. With regard to Figure 4 and Figure 5 An example 2D ultrasound image including an indicator of the pleural line is described.

[0054] Go to briefly Figure 4 , shows an example annotated 2D lung ultrasound image 400, which may be output to a display, such as with respect to Figure 3 . The example annotated 2D lung ultrasound image 400 includes a 2D lung ultrasound image 402 that may be generated at operation 306 of the method 300, and a pleural line indicator 404 that may be generated at operation 308 of the method 300. The pleural line indicator 404 may be positioned on a bottom edge (e.g., a lower border) of the pleural line, and may visually indicate pixels of the 2D lung ultrasound image 402 that are identified as the pleural line. For example, the pleural line 406 is indicated by the pleural line indicator 404. In other examples, the pleural line 406 may be depicted (e.g., with a continuous line) to visually indicate the pleural line 406 in the annotated 2D lung ultrasound image 400.

[0055] The vertical position of each of the pleural line indicators 404 can be different and can reflect curvatures, protrusions, cavities, and / or other irregularities in the pleural line. For example, the first pleural indicator 410 can have a higher vertical position relative to the second pleural indicator 412. The orientation of the 2D lung ultrasound image 402 is such that the outside of the patient is toward the top of the image and the inside of the patient's lungs is toward the bottom of the image. Ultrasound imaging data vertically below the pleural line 406 (e.g., below each of the pleural line indicators 404 and the space between them) can be noise and, therefore, may not indicate the topography of the pleural line.

[0056] Return to Figure 3 After identifying the bottom edge, at 310, method 300 includes removing ultrasound data from below the bottom edge of the pleura in each ultrasound image. Figure 4 As described, ultrasound imaging data vertically below the bottom edge of the pleural line may be noise that does not indicate the topography of the pleural line. As described above, subpleural consolidation is included in the boundary defined by the bottom edge. Removing ultrasound data from below the bottom edge may expose the topography of the pleural line. Removing ultrasound data may include generating a new data set for each ultrasound image, the new data set including ultrasound data above and including the pleural line, and not including data (e.g., noise) below the bottom edge of the pleural line.

[0057] At 312, method 300 includes generating a 3D pleural surface based on the 2D ultrasound image. In some examples, operation 312 may be performed before operation 310 (eg, removing ultrasound data from below the bottom edge of the pleural line), as described with respect to Figure 5 Further described. Two methods for generating a 3D pleural surface are described herein, however, additional and / or alternative methods for generating a 3D pleural surface from a 2D ultrasound image may be used without departing from the scope of the present disclosure. The two methods described herein generate a 3D pleural surface from volume ultrasound data. At 314, method 300 includes stacking a series of 2D ultrasound images to generate a 3D pleural surface. For example, the series of 2D ultrasound images may include more than 100 2D ultrasound images generated from data captured by positioning a 2D probe on different overlapping or non-overlapping areas of a patient region and acquiring ultrasound data according to a lung ultrasound protocol. Each 2D ultrasound image in the series of 2D ultrasound images may capture a different view of the same patient region, so that the topography of the pleural line (e.g., a protrusion protruding into the lung, a cavity extending away from the lung) can be shown from different perspectives in each 2D ultrasound image. Stacking the series of 2D ultrasound images may include aligning the pleural line shown in each 2D ultrasound image (e.g., aligning the corresponding pleural line indicator 404 in each image) to simulate a 3D ultrasound image.

[0058] Go to Figure 5 , an example method 500 for stacking a series of 2D ultrasound images to generate a 3D pleural surface is shown. A 2D ultrasound image 502 is shown as an example 2D ultrasound image in a series of 2D ultrasound images. The 2D ultrasound image 502 includes a pleural line indicator 404 indicating a pleural line 504. Ultrasound data from below the bottom edge of the pleural line 504 has not been removed, so the 2D ultrasound image 502 includes noise 506 below the pleural line 504. It should be understood that other 2D ultrasound images in the series of 2D ultrasound images similarly include pleural line indicators, and ultrasound data from below the pleural line has not been removed. The stack 510 of 2D ultrasound images includes a series of 2D ultrasound images capturing the same patient area from different views, wherein the 2D ultrasound image 502 is at the top of the stack. Each of the 2D ultrasound images below the 2D ultrasound image 502 can be aligned with the 2D ultrasound image 502 along the pleural line 504. For example, the corresponding pleural line indicators 404 can be aligned in space (e.g., the same x, y, and z coordinates relative to the reference axis system 512). In one example, the y-axis can be a vertical axis (e.g., parallel to the gravity axis), the x-axis can be a lateral axis (e.g., a horizontal axis), and the z-axis can be a longitudinal axis. However, in other examples, the axes can have other orientations. In order to form the 3D pleural surface 514, the stack 510 of 2D ultrasound images can be processed to convert the series of 2D ultrasound images 510 into a single continuous 3D image. In addition, ultrasound data (e.g., noise) from below the bottom edge of the pleural line 504 can be removed to expose the pleural surface 516. The pleural surface 516 is shown by looking from the inside of the lungs outward. For example, the outside of the patient is toward the top of the image, and the inside of the patient's lungs are toward the bottom of the image. As shown in FIG. Figure 5 As shown, the pleural surface 516 may include protrusions, such as protrusion 518 that protrudes into the inside of the lung. Figure 7 As further described, the protrusion 518 can be indicated by comparing the pleural line indicator 404. Figure 3 and Figures 7 to 11 As further described, the 3D pleural surface 514 may be shaded by at least one virtual light source to highlight protrusions of the pleural surface 516 .

[0059] Return to Figure 3At 316, method 300 includes generating a 3D mesh from a series of 2D ultrasound images and generating a 3D pleural surface using the 3D mesh. Generating the 3D mesh may include identifying coordinate points of the pleural line in space, for example relative to an x, y, and z axis system, in each 2D ultrasound image in the series of 2D ultrasound images. For example, each 2D ultrasound image in the series of 2D ultrasound images may have the same size (e.g., the same length, width, number of pixels, etc.), however, due to the positioning of the 2D probe to acquire ultrasound data as described above, the coordinate points of the pleural line may be different in different images in the series of 2D ultrasound images. Each 2D ultrasound image may be positioned at the same position on the axis system (e.g., the lower right corner is at (0,0)), and the coordinates of each pleural line indicator may be identified. As described with respect to Figure 6 As further described, the 3D grid may be constructed using the coordinates of each corresponding pleural line indicator for each 2D ultrasound image in the series of 2D ultrasound images.

[0060] Go to Figure 6 , an example method 600 for generating a 3D pleural surface from a 3D mesh based on a series of 2D images is shown. A first 2D ultrasound image 602 and a second 2D ultrasound image 604 may be included in the series of 2D ultrasound images. Each of the first 2D ultrasound image 602 and the second 2D ultrasound image 604 includes a pleural line indicator 404 indicating a pleural line 504. Ultrasound data from below the pleural line 504 has not been removed in any image, so the first 2D ultrasound image 602 and the second 2D ultrasound image 604 include noise 506 below the pleural line 504. It should be understood that other 2D ultrasound images in the series of 2D ultrasound images similarly include pleural line indicators, and ultrasound data from below the pleural line has not been removed.

[0061] As about Figure 3 As described, each of the first 2D ultrasound image 602 and the second 2D ultrasound image 604 may have the same size (e.g., length and width) and may capture different views of a lung region (e.g., the pleural line 504). The first 2D ultrasound image 602 and the second 2D ultrasound image 604 are each aligned with the reference axis system 512. For example, the lower left corner of each image is located at (0,0). In order to generate a 3D grid, the pleural line 504 in the first 2D ultrasound image 602 is mapped to the pleural line 504 in the second 2D ultrasound image 604. For example, the first pleural indicator 620, the second pleural indicator 622, and the third pleural indicator 624 of the first 2D ultrasound image are each mapped to the fourth pleural indicator 630, the fifth pleural indicator 632, and the sixth pleural indicator 634 of the second 2D ultrasound image 604, respectively. Figure 6A comparison of two 2D ultrasound images is shown for illustrative purposes, however generating the 3D mesh may include comparing more than two 2D ultrasound images in the series of 2D ultrasound images. For example, the 3D mesh may be generated based on a comparison of at least 100 2D ultrasound images in the series of 2D ultrasound images. A 3D mesh, such as a mesh comprised of polygons, wherein the height, width, and depth of the polygons are defined by coordinate points of the pleural line 504 in the series of 2D ultrasound images being compared. A 3D pleural surface 514 may be generated from the 3D mesh, wherein one or more 2D ultrasound images in the series of 2D ultrasound images are deformed according to the topography of the 3D mesh. Thus, the 3D pleural surface 514 includes the pleural line 504 and protrusions of the pleural line 504, such as protrusion 518. As described with respect to Figure 5 As depicted, the pleural surface 516 is shown looking outward from the inside of the lung, and the 3D pleural surface 514 may be shaded by at least one virtual light source to highlight protrusions of the pleural surface 516 .

[0062] Return to Figure 3 At 318, method 300 includes saving the 3D pleural surface to a memory and outputting the 3D pleural surface for display on a display device. In some examples, the display is included in an ultrasound imaging system, such as display device 118. The 3D pleural surface may or may not be displayed with an annotation indicating the pleural line. Figures 8 to 10 An example display of a 3D pleural surface is shown in . In some examples, the 3D pleural surface can be saved with and without annotations (e.g., pleural line indicators). In addition, at least in some examples, the original unprocessed ultrasound data can be saved. The memory can be local to the ultrasound imaging system, or can be a remote memory. For example, unannotated images and annotated images can be saved and / or archived (e.g., as a structured report in a PACS system) so that they can be retrieved and used to generate a formal physician-signed report that can be included in the patient's medical record (e.g., EHR).

[0063] Therefore, compared with 2D ultrasound images of the pleura, Figure 3 The 3D pleural surface generated as described includes a more detailed view of the pleura. A series of 2D ultrasound images (e.g., a series of 2D ultrasound images used to generate a 3D pleural surface) can be used to visualize the topography of the pleura shown in a single 3D pleural surface. Therefore, generating a 3D pleural surface can simplify image analysis by showing the detailed topography of the pleura in one image (e.g., a 3D pleural surface) rather than in multiple images (e.g., a series of 2D ultrasound images).

[0064] As about Figure 5 and Figure 6Briefly described, at 320, method 300 optionally includes applying a shadow to the 3D pleural surface. The 3D pleural surface may be shaded via at least one virtual light source positioned as if it were inside the lung. The shadow may highlight protrusions extending from the 3D pleural surface toward the inside of the lung (e.g., Figure 5 and Figure 6 For example, at least one virtual light source may be positioned at a preset position within the inner side of the lung, such as at an acute angle relative to the 3D pleural surface and / or a protrusion protruding from the 3D pleural surface. The position, brightness, color, and / or number of virtual light sources may be adjusted in response to receiving user input. For further details on shading the 3D pleural surface, refer to Figure 7 Give a description.

[0065] Figure 7 An example is shown for a 3D pleural surface (e.g., as described with respect to Figure 3 and / or Fig.11 Flowchart of an example method 700 for shading a 3D pleural surface generated by the method described above. Specifically, method 700 provides a workflow for positioning at least one virtual light source to point to a 3D pleural surface as if it were on the inside of a lung (e.g., from the inside of the lung to the outside of the lung). Method 700 may be implemented by one or more of the above systems, including Figure 1 The ultrasound imaging system 100 and Figure 2 Therefore, the method 700 can be stored as executable instructions in a non-transitory memory (such as Figure 1 The memory 120 and / or Figure 2 206) and by a processor such as Figure 1 The processor 116 and / or Figure 2 processor 204) to execute.

[0066] At 702, method 700 includes identifying a pleural irregularity of the pleura. For example, the pleural irregularity can be a protrusion extending from the 3D pleural surface toward the inside of the lung, or a cavity extending into the 3D pleural surface of the pleura (e.g., from the bottom edge of the pleura, away from the lung). Method 700 can be applied to Figure 3 An example of a 3D pleural surface generated by the method 300 may be Figure 5 and Figure 6 3D pleural surface 514. For example, the processor may evaluate each pixel of the identified pleura (e.g., within the upper and lower boundaries of the pleural line) to locally characterize the pleura as healthy or irregular by a predetermined scoring criterion. Figure 3As described, detecting the pleural position may include identifying the lower and upper boundaries of the pleura based on brightness changes between pixels, such as by using edge detection techniques or gradient changes. The processor may evaluate each pixel of the pleural line to determine a beating score and a dimness score for each pleural position in order to identify the location of the pleural irregularity. The beating score evaluates the vertical position of the pleural line at each horizontal position to identify a vertical gap in the pleural line, wherein a larger vertical gap results in a higher beating score. For example, a vertical gap may refer to a plurality of pixels located vertically between the lower boundary (or upper boundary) of the pleural line relative to adjacent pixels at a given pixel position. A vertical gap between the upper or lower boundary of the pleural line at adjacent horizontal positions may cause the pleural line to have, for example, a discontinuous or rough appearance. The dimness score ranks the brightness (or dimness) of the pleural pixels at a specific horizontal position relative to its adjacent pixels. As the local pixel brightness of the pleura decreases relative to its adjacent pixels (e.g., the pixel becomes darker relative to its adjacent pixels), the dimness score increases.

[0067] The irregularity score for each pixel along the pleural line in each frame can be generated as the product of the jitter score and the dimness score and compared to a threshold score. The threshold score can be a predetermined value stored in a memory that distinguishes irregular pleura associated with a disease state from normal healthy pleura. In some examples, the threshold score can be adjusted based on curated data and using a support vector machine. If the irregularity score is greater than or equal to the threshold score, the pleura imaged in the pixel position can be considered irregular. In contrast, if the irregularity score is less than the threshold score, the pleura imaged in the pixel position may not be considered irregular (e.g., may be considered normal and / or healthy). Although the pleura can be analyzed on a pixel-by-pixel basis, a filter can be used to smooth the results. Therefore, pixel regions of a predetermined size can be grouped, and the pixel region can be identified as the location of an irregular portion (e.g., irregular pleura) in response to the majority (e.g., greater than 50%) of the pixels within the group being characterized as irregular pleura (e.g., protrusions). In contrast, a region of pixels may be identified as healthy in response to a majority of pixels within the group being characterized as healthy pleura.

[0068] After identifying the pleural irregularity (eg, a protrusion), at 704, method 700 includes positioning the virtual light source as if it were inside the lung. Figures 3 to 6As described, the 3D pleural surface is viewed from the inside of the lung and looks outward (e.g., the area below the pleural surface 516 in the 3D pleural surface 514 is considered to be on the inside of the lung). The virtual light source may be positioned so that when illuminated, the virtual light source directs light toward the pleural surface 516. The virtual light source may be positioned in a preset position relative to the pleural surface, such as the lower right, upper left, lower left, or upper right of the display. In some examples, the virtual light source may be positioned relative to the identified protrusion. For example, when the identified protrusion is in the upper left quadrant of the display, the virtual light source may be positioned in the lower right quadrant of the display. At 706, method 700 includes highlighting the pleural irregularity by illuminating the virtual light source to a first brightness. For example, the virtual light source may emit light radially, linearly, or as a ray, and as further described herein, the shape of the emitted light may be adjusted in response to user input. As further described herein, the brightness and color / hue of the virtual light source may be further adjusted in response to user input.

[0069] Go to briefly Figure 8 , illustrates a first example 3D lung model 800 output to a display 801. The display 801 may be, for example Figure 1 The display device 118. Figure 7 As further described, the display 801 includes sliders for adjusting the brightness (e.g., brightness slider 810), color / hue (e.g., hue box 812), and position (e.g., coordinate selector 814) of one or more light sources, as well as a reference axis system 890. In one example, the y-axis can be a vertical axis (e.g., parallel to the gravity axis), the x-axis can be a lateral axis (e.g., a horizontal axis), and the z-axis can be a longitudinal axis. However, in other examples, the axes can have other orientations.

[0070] The first example 3D lung model 800 includes a throat (e.g., trachea) 802, a left lung 804, and a right lung 806. The left lung 804 is illustrated as a cross-section (e.g., taken along the yx plane relative to the reference axis 890) showing the inner side 808 and the pleura 816 of the left lung 804. In other embodiments of the 3D lung model 800, the right lung 806 may additionally or alternatively be illustrated as a cross-section. A 3D pleural surface (e.g., 3D pleural surface 514) is superimposed on the left lung 804 in an approximate anatomical position and orientation such that the pleural line of the 3D pleural surface is aligned with the pleura 816 of the left lung 804. For example, the 3D pleural surface 514 can be generated from imaging data of the left lung 804 captured at the approximate position where the 3D pleural surface is superimposed on the left lung 804 in the display 801. The pleural surface 516 is shown as being exposed to the interior of the left lung 804. Although in Figure 8Only one 3D pleural surface is shown in the example of FIG, but additional generated 3D pleural surfaces may be positioned at different locations of the first example 3D lung model 800, such as at the back of the left lung 804, at the side of the left lung 804, at the front of the right lung 806, at the back of the right lung 806, or at the side of the right lung 806. In addition, although Figure 8 Not explicitly shown in , but the 3D pleural surface may include annotations as described above.

[0071] As about Figure 7 As described, the virtual light source 820 can be positioned within the inner side 808 of a lung (e.g., the left lung 804) and used to shade the 3D pleural surface 514. In some examples, the virtual light source 820 can be automatically positioned at a preset or random coordinate within the inner side of the lung corresponding to the 3D pleural surface 514. The coordinates of the virtual light source can be shown in the coordinate selector 814. The virtual light source 820 can be positioned relative to a protrusion identified in the 3D pleural surface 514 (e.g., which extends toward the inner side 808 of the lung). For example, as described with respect to Figure 7 As described, one or more protrusions of the 3D pleural surface 514 may be identified, and the virtual light source 820 may be positioned at coordinates where the virtual light source 820 and a light beam (e.g., indicated by arrow 822) projected by the virtual light source 820 are at an acute angle relative to the 3D pleural surface 514. In this way, the virtual light source 820 may highlight the protrusions extending from the pleural surface 516 toward the inner side 808 of the left lung 804. This may help a user viewing the display 801 (such as a healthcare provider or imaging technician) to quickly and accurately identify pleural irregularities in the 3D pleural surface, which may help reduce diagnostic timelines and increase diagnostic accuracy.

[0072] Go to Fig. 9 , such as about Figure 8 As described, a second example 3D lung model 900 output to the display 801 is illustrated. The second example 3D lung model 900 may include a 3D pleural surface (e.g., 3D pleural surface 514) centered on the quadrant grid and oriented such that the outside of the patient is located in the upper right quadrant and the upper left quadrant, and the inside of the patient's lungs are located in the lower right quadrant and the lower left quadrant of the quadrant grid. Automatically positioning the virtual light source 820 may include identifying protrusions (e.g., about Figure 7), and positions the virtual light source 820 in the complementary quadrant. For example, when the protrusion 518 is in the lower left quadrant, the virtual light source 820 can be automatically positioned in the lower right quadrant so that the light beam indicated by the arrow 822 highlights at least a portion of the protrusion 518. In an example where the protrusion is in the lower right quadrant, the virtual light source 820 can be positioned in the lower left quadrant. In some examples, the virtual light source 820 can be automatically positioned without identifying the protrusion. The virtual light source 820 can be positioned at a preset or random coordinate in one of the lower right quadrant or the lower left quadrant. As described with respect to Figure 7 As further described, the virtual light source 820 can be moved in response to user input, such as changing the coordinates of the virtual light source 820 in the coordinate selector 814, selecting and moving the virtual light source 820 (e.g., dragging a selection tool on the display 801), selecting a new position for the virtual light source 820 on the display 801, etc. Figure 8 As described, the second example 3D lung model 900 may help a user to quickly and accurately identify pleural irregularities in the 3D pleural surface.

[0073] Go to Fig.10 , such as about Figure 8 and Fig. 9 As described, an example lung model 1000 is illustrated as being output to the display 801. The example lung model 1000 may include a 3D pleural surface (e.g., the 3D pleural surface 514) and a representative 2D image (e.g., the 2D ultrasound image 502) used to generate the 3D pleural surface, which may be simultaneously displayed in the image window 1002. Fig. 9 As described, the 3D pleural surface 514 can be centered on the quadrant grid and oriented so that the outside of the patient is located in the upper right and upper left quadrants, and the inside of the patient's lungs are located in the lower right and lower left quadrants of the quadrant grid. The 3D pleural surface 514 can be a live 3D surface rendering of the detected pleura, or can be a static image provided after data collection using an ultrasound probe. The virtual light source 820 can be as described with respect to Figures 7 to 9 2D ultrasound image 502 may be included in the example lung model 1000 to illustrate the generation of a 3D pleural surface 514 therefrom (e.g., as described with respect to Figure 3 and Fig.11 ), which can help identify and diagnose pleural irregularities. For example, the 2D ultrasound image 502 can be a live B-mode slice, which can be automatically selected or selected by the user through interaction with a user interface (e.g., user interface 115).

[0074] Return to Figure 7In some examples, method 700 may include receiving user input at 708 and adjusting the displayed content accordingly. For example, the user input may include inputting x, y, and / or z coordinates (e.g., inputting to Figures 8 to 10 814) to update the position of the virtual light source. In some examples, the user can select the virtual light source on the display (e.g., click, drag and release, or select the current position and then select a new position). The user input can additionally or alternatively include adjustments to the brightness of the virtual light source, such as selecting a new brightness from the brightness options, entering a new brightness, using a slider (e.g., Figures 8 to 10 The user input may include adjusting the color / hue of the virtual light source, for example, from white light to colored light or adjusting the amount of red, blue, and / or green light (e.g., using the hue box 812). The user input may include adding one or more additional virtual light sources, which may have a default position, color / hue, brightness, etc. when added. The characteristics of the additional virtual light sources (e.g., position, color / hue, brightness, etc.) may be adjusted individually as described above. At 710, the method 700 includes adjusting at least one of the virtual light source position, color / hue, brightness, etc. based on the user input.

[0075] After making adjustments based on the user input, at 712, method 700 may include updating the virtual light source positioning parameters. For example, method 700 may use an algorithm to automatically position the virtual light source before receiving the user input. After receiving the user input, the algorithm may be updated to modify the initial positioning of the virtual light source based on the user input. For example, the algorithm may learn common user selections of parameters including color / hue, brightness, position, etc., and apply these parameters when automatically positioning the virtual light source during the following specific implementation of method 700.

[0076] Thus, method 700 can be performed according to Figure 3 and Fig.11 The 3D pleural surface generated by the method provides a shadow that highlights pleural irregularities (such as protrusions protruding to the inner side of the lung), making the pleural irregularities more visible on the display compared to the unshaded 3D pleural surface. The mental burden of the operator can be reduced. In addition, the variability of the accuracy and frequency of pleural irregularity detection between operators is reduced. Overall, the accuracy of diagnosis can be improved, and the amount of time before diagnosis can be reduced.

[0077] Fig.11 A method for generating a 3D pleural surface (eg, Figure 5 to Figure 6 and Figures 8 to 1014) in which ultrasound imaging signals for generating a 3D pleural surface are acquired using a probe capable of live 3D imaging (such as a matrix array probe or other volume probe) to acquire a 3D volume. Specifically, method 1100 provides a workflow for generating a 3D pleural surface from a 3D image captured using a probe capable of live 3D imaging by identifying a pleural line in the 3D image and segmenting the 3D image at the pleural line to expose the 3D pleural surface. Figure 1 Method 1100 is described using a 3D ultrasound image acquired by an ultrasound imaging system 100 of the type described above, but other ultrasound imaging systems may also be used. In addition, method 1100 may be applicable to other imaging modalities. Method 1100 may be implemented by one or more of the above systems, including Figure 1 The ultrasound imaging system 100 and Figure 2 Therefore, the method 1100 can be stored as executable instructions in a non-transitory memory (such as Figure 1 The memory 120 and / or Figure 2 206) and by a processor such as Figure 1 The processor 116 and / or Figure 2 In addition, in some embodiments, the method 1100 is performed in real time while the 3D image is being acquired, while in other embodiments, at least a portion of the method 1100 is performed offline after the 3D image is acquired. For example, the processor may evaluate the 3D image stored in the memory even when the ultrasound system is not actively operated to acquire images.

[0078] At 1102, method 1100 includes receiving a lung ultrasound protocol selection. The lung ultrasound protocol may be selected by an operator (e.g., a user) of the ultrasound imaging system through a user interface (e.g., user interface 115). As an example, the operator may select a lung ultrasound protocol from a plurality of possible ultrasound protocols using a drop-down menu or by selecting a virtual button. Alternatively, the system may automatically select a protocol based on data received from an EHR associated with the patient. For example, the EHR may include previously performed examinations, diagnoses, and current treatments, which may be used to select a lung ultrasound protocol. In addition, in some examples, the operator may manually input and / or update parameters for a lung ultrasound protocol. The lung ultrasound protocol may be a system-guided protocol in which the system guides the operator step by step through the protocol, or may be a user-guided protocol in which the operator follows a laboratory-defined or customized protocol, and the system does not enforce a specific protocol or does not know the protocol steps in advance.

[0079] In addition, the lung ultrasound protocol may include multiple scanning sites (e.g., views), probe movements, and / or imaging modes that are performed sequentially. For example, the lung ultrasound protocol may include the use of a dynamic M-mode. The lung ultrasound protocol may include a longitudinal scan, in which the probe is positioned perpendicular to the ribs, and / or an oblique scan, in which the probe is positioned along the intercostal spaces between the ribs. In addition, in an example in which the ultrasound probe is a matrix probe, the lung ultrasound protocol may include indicating a static position on the patient at which the ultrasound probe is maintained while the volume ultrasound data is captured by the ultrasound probe.

[0080] At 1104, method 1100 includes collecting ultrasound data with an ultrasound probe according to a lung ultrasound protocol by transmitting and receiving ultrasound signals. Collecting ultrasound data according to the lung ultrasound protocol may include the system displaying instructions on a user interface, for example, to guide an operator to complete the collection of a specified scanning site. Additionally or alternatively, the lung ultrasound protocol may include instructions for the ultrasound system to automatically collect some or all data or perform other functions. For example, the lung ultrasound protocol may include instructions for the user to position, move, rotate, tilt and / or swing the ultrasound probe and automatically start and / or terminate the scanning process and / or adjust the imaging parameters of the ultrasound probe, such as ultrasound signal transmission parameters, ultrasound signal reception parameters, ultrasound signal processing parameters, or ultrasound signal display parameters. In some embodiments, live 3D image data may be collected by a matrix 2D probe or by a mechanically swung 1D probe. In addition, the acquired ultrasound data includes one or more image parameters calculated for each pixel or a group of pixels to be displayed (e.g., a group of pixels assigned the same parameter value), wherein the one or more calculated image parameters include, for example, one or more of intensity, velocity, color blood flow velocity, texture, granularity, contractility, deformation, and deformation rate values.

[0081] At 1106, the method 1100 includes generating a 3D ultrasound image from the acquired ultrasound data. For example, the signal data acquired at 1104 during the method is processed and analyzed by a processor to generate a 3D ultrasound image at a specified frame rate. The processor may include an image processing module that receives the signal data (e.g., image data) acquired at 1104 and processes the received image data. For example, the image processing module may process the ultrasound signal to generate a 3D volume rendering of ultrasound information for display to an operator. In one example, generating the 3D ultrasound image may include determining an intensity value for each pixel to be displayed based on the received image data.

[0082] At 1108, method 1100 includes generating a 3D pleural surface from the 3D ultrasound image. Generating the 3D pleural surface can be performed in two steps. At 1110, method 1100 includes detecting a bottom edge of the pleura in the 3D ultrasound image. In an inflated lung, the pleura, which forms an outer boundary of the lung against the chest wall, may provide essentially the only anatomical lung structure that can be detected by ultrasound. The pleura appears in the 3D ultrasound image as a high-echo level segment of brighter (e.g., whiter) pixels, called the pleural line, which moves synchronously with breathing in a phenomenon called pleural sliding. Detecting the pleural position may include identifying a lower boundary and an upper boundary of the pleura based on brightness changes between pixels, such as by using edge detection techniques or gradient changes. For example, the processor may apply a method such as included in Figure 2 An edge detection algorithm in the 3D generation module 212 of the processor may include one or more mathematical methods for identifying points (e.g., pixels) where the image brightness changes sharply and / or has discontinuities to identify the lower and upper boundaries of the pleural line. As an example, the processor may apply the edge detection algorithm to the area with the highest local variation. As another example, additionally or alternatively, the gradient algorithm may identify the local maximum and / or minimum pixel brightness at the pleural location to identify the lower and upper boundaries of the pleural line in each image. In frames where there is no pleural sliding (e.g., between breaths), the position of the pleura may be tracked up / down based on the known position of the pleura from previous frames.

[0083] The lower and upper boundaries of the pleura may include subpleural consolidation. For example, the bottom edge (e.g., as further described herein, indicating a boundary below which ultrasound data is removed) may be positioned at a predetermined distance from the pleural line toward the lung. It will be appreciated that, conventionally, subpleural consolidation extends a maximum distance 'n' from the pleural surface toward the lung. Therefore, the bottom edge of the pleura may be positioned at a distance 'n' from the lower boundary of the pleura identified as described above. In other embodiments, the same or different edge detection algorithms as described above may be used to identify subpleural consolidation.

[0084] After identifying the pleural location, at 1112, method 1100 includes removing ultrasound data from below a bottom edge of the pleural line in the 3D ultrasound image. Figure 4 As described, ultrasound imaging data vertically below the bottom edge of the pleural line may be noise that is not indicative of the topography of the pleural line. Removing ultrasound data from below the bottom edge of the pleural line may expose the topography of the pleural line. Removing ultrasound data may include generating a new data set for each ultrasound image, the new data set including ultrasound data above and including the pleural line, and not including data (e.g., noise) below the bottom edge of the pleural line.

[0085] Go to briefly Fig.12, an example image set 1200 including a 3D ultrasound image 1202 and a 3D pleural surface 1204 is shown. The 3D ultrasound image 1202 may be an example of a 3D ultrasound image generated (e.g., at operation 1106 of method 1100) from ultrasound data acquired using a volume probe. As described with respect to method 1100, the 3D pleural surface 1204 may be generated from the 3D ultrasound image. Each of the 3D ultrasound image 1202 and the 3D pleural surface 1204 may include a pleural line indicator 1206 that may be generated at operation 1108 of method 1100. The pleural line indicator 1206 may be positioned on a lower boundary (e.g., bottom edge) of the pleural line and may visually indicate pixels of the 3D ultrasound image 1202 that are identified as the pleural line. In other examples, the pleural line may be depicted (e.g., with a continuous line) to visually indicate the pleural line in the 3D ultrasound image 1202. Removing ultrasound data from below the pleural line may include removing ultrasound data below the pleural line indicators 1206. Because the vertical position of each of the pleural line indicators 1206 may be different and may reflect curvatures, protrusions, and / or other irregularities in the pleural line, removing ultrasound data below the pleural line indicators 1206 produces a topographic pleural surface 1216 that includes protrusions, such as protrusion 1218, that protrude to the inside of the lung.

[0086] Return to Fig.11 After generating the 3D pleural surface at operation 1108, at 1114, method 1100 includes outputting the 3D pleural surface for display on a display device. In some examples, the display is included in the ultrasound imaging system, such as display device 118. The 3D pleural surface may or may not be displayed with annotations indicating the pleural line. Figures 8 to 10 An example display of a 3D pleural surface is shown in . In some examples, the 3D pleural surface can be saved with and without annotations (e.g., pleural line indicators). In addition, at least in some examples, the original unprocessed ultrasound data can be saved. The memory can be local to the ultrasound imaging system, or can be a remote memory. For example, unannotated images and annotated images can be saved and / or archived (e.g., as a structured report in a PACS system) so that they can be retrieved and used to generate a formal physician-signed report that can be included in the patient's medical record (e.g., EHR).

[0087] At 1116, method 1100 optionally includes applying a shadow to the 3D pleural surface. The 3D pleural surface may be shaded via at least one virtual light source positioned as if it were inside the lung. The shadow may highlight protrusions extending from the 3D pleural surface toward the inside of the lung (e.g., Fig.12 At least one virtual light source may be positioned to apply shadows to the Figure 7 For example, at least one virtual light source may be positioned at a preset position inside the inner side of the lung, such as at an acute angle relative to the 3D pleural surface and / or a protrusion protruding from the 3D pleural surface. The position, brightness, color and / or number of the virtual light sources may be adjusted in response to receiving user input.

[0088] Compared with 2D ultrasound images of the pleura, Fig.11 The 3D pleural surface generated as described includes a more detailed view of the pleura. When generating a 3D pleural surface from volumetric ultrasound data captured using a volumetric probe, the same image as that used to capture the pleural surface for generating the 3D pleural surface (e.g., as described with respect to Figure 3 Compared with the number of scans of a series of 2D ultrasound images (described above), the number of scans performed to capture volumetric ultrasound data can be reduced. Therefore, when the volumetric data is acquired by the ultrasound probe, the processing power requirements of the image processing system can be reduced.

[0089] Can be expanded on Figure 3 , Figure 7 and Fig.11 The described method for generating a 3D pleural surface provides further automatic identification and highlighting of pleural surface features and irregularities. For example, a method can be used to perform surface-based scoring of irregularities and B-lines indicating the potential severity of pleural irregularities, which can assist in patient diagnosis. In addition, a conventional automatic B-line algorithm can be extended to a 3D pleural surface and used to identify and display B-lines of a 3D pleural surface so that the B-lines can be displayed in a 3D graphic. For example, this may include volume rendering of B-lines based on detected pleural irregularities (e.g., protrusions from the pleural surface toward the inside of the lung). The conventional automatic B-line algorithm can be further extended to other lesions (such as consolidation and pleural effusion) that can be shown in a 3D pleural surface. In addition, the generated 3D pleural surface can be used to identify pneumothorax (PTX). For example, a lung point (e.g., the edge of a PTX) is shown as a contour of the 3D pleural surface, which can be used with respect to Figures 7 to 10 The virtual light source described is highlighted.

[0090] Fig.13 A fourth example display 1300 is shown, including biplane and 3D images of the lungs (including the pleural surface). Methods utilizing existing 3D ultrasound imaging techniques (such as those described with respect to Fig.11The method 1100 described above is used to obtain an image of the fourth example display content 1300. The orientation view 1302 shows a 3D imaging region 1308 (e.g., lungs), and includes a first plane 1304 and a second plane 1306 indicating a bi-plane cross-section of the 3D imaging region 1308. The first 2D plane view 1310 shows the 3D imaging region 1308 segmented by the first plane 1304. The second 2D plane view 1320 shows the 3D imaging region 1308 segmented by the second plane 1306. The first 2D plane view 1310 and the second 2D plane view 1320 are segmented by different planes to show different regions of the lungs, including different regions of the pleural surface.

[0091] Each of the first 2D plan view 1310 and the second 2D plan view 1320 includes a line 1312, which is generally used to cut the 2D image to realize the visualization of the topography of the 3D pleural surface. For example, line 1312 is shown as a linear line, but the cutting of the 3D pleural surface can be nonlinear, and the 3D topography of the 3D pleural surface is shown alternatively, including protrusions and cavities. The first 2D plan view 1310 and the second 2D plan view 1320 also include a B line, which can indicate irregular parts in the pleural surface, such as protrusions and / or cavities. Linear white line 1314 can indicate pleural lines. In the first 2D plan view 1310, the first B line 1316 extends from the linear white line 1314 toward line 1312. In the second 2D plan view 1320, the second B line 1318, the third B line 1320 and the fourth B line 1322 all extend toward line 1312. In some examples, third B-line 1320 may represent the same pleural irregularity represented by first B-line 1316 , as viewed from a different viewing angle.

[0092] The first arrow 1324 and the second arrow 1326 may indicate the location and direction of one or more virtual light sources used to highlight the 3D pleural surface image 1330. The one or more virtual light sources may be as shown in FIG. Figure 7 Positioned as described.

[0093] For example, regarding Figure 3 and Fig.11As described, a 3D pleural surface image 1330 can be formed by a 3D imaging region 1308 by removing ultrasound data including noise and non-pleural surface topography. The 3D pleural surface image 1330 shows the topography of the pleural surface, including the protrusions and cavities of the pleural line. In the first 2D plane view 1310 and the second 2D plane view 1320, these protrusions and cavities of the pleural line are shown as B lines. However, although pleural irregularities and other lesions are visible in the 2D plane view, some characteristics of pleural irregularities and other lesions may not be included due to the 2D nature of the 2D plane view. Compared with the 3D pleural surface image 1330, it may be challenging for an observer of the 2D plane view to understand the entire pleural surface (including the 3D topography of the pleural line and its irregularities). For example, the protrusions protruding from the pleural surface enclosed in the circle 1332 can be indicated in the first 2D plane view 1310 and the second 2D plane view 1320, respectively. The 3D pleural surface image 1330 shows the protrusions in greater detail, which can aid in the identification and diagnosis of pleural irregularities and other pathologies.

[0094] In this way, the processor can automatically generate a 3D pleural surface of the lung pleura viewed from the inside of the lung and looking outward based on the ultrasound imaging signal. The method and system described herein realize the visualization of the entire pleural surface captured by the ultrasound probe, which is contrary to the representative 2D image, and the representative 2D image may or may not include lesions existing in the 3D space and / or extending into the 3D space. This can eliminate the diagnostic step of searching for a specific view of the pleural surface. In this way, a wide range of ultrasound findings and lesions can be visualized, including pleural irregularities, pneumothorax, and viral and bacterial infections. Therefore, the amount of time spent by health care professionals viewing medical images can be reduced, allowing health care professionals to focus on patient care and comfort. In addition, by including a 3D pleural surface superimposed on a 3D rendering of a lung model, pleural irregularities can be displayed in an anatomically relevant environment to further simplify the diagnostic process.

[0095] A technical effect of generating a 3D pleural surface of the lung pleura viewed from the inside of the lung and looking outward is that the processing power used by the imaging system can be reduced due to a reduction in the number of additional imaging scans due to the increased accuracy and detail of the pleural surface image rendering.

[0096] The present disclosure also provides support for a method for imaging a patient's lungs, the method comprising: generating a three-dimensional (3D) pleural surface viewed from the inside of the lungs and looking outward based on ultrasound imaging signals. In a first example of the method, ultrasound imaging signals are captured by positioning a one-dimensional (1D), 1.5-dimensional (1.5D) or matrix (two-dimensional, 2D) probe to acquire a 2D image. In a second example of the method, optionally including the first example, ultrasound imaging signals are captured by positioning a probe capable of live 3D imaging to acquire volume data. In a third example of the method, optionally including one or both of the first and second examples, the method further comprises: generating an ultrasound image from the ultrasound imaging signal, detecting the bottom edge of the pleura in the ultrasound image, and removing ultrasound data from below the bottom edge of the pleura. In a fourth example of the method, optionally including one or more or each of the first to third examples, the method further comprises: shading the 3D pleural surface via at least one virtual light source, the at least one virtual light source being positioned as if at an acute angle relative to the 3D pleural surface on the inside of the lung. In a fifth example of the method, optionally including one or more or each of the first to fourth examples, shading includes: highlighting a pleural irregularity of the 3D pleural surface, and positioning the at least one virtual light source within the inner side of the lung at the acute angle relative to the pleural irregularity. In a sixth example of the method, optionally including one or more or each of the first to fifth examples, a user controls the position of the at least one virtual light source via the display of the 3D pleural surface. In a seventh example of the method, optionally including one or more or each of the first to sixth examples, the method further includes: identifying and displaying a B-line of the 3D pleural surface.

[0097] The present disclosure also provides support for a system, the system comprising: a display device; and a processor, the processor being configured with instructions in a non-transitory memory, which, when executed, causes the processor to: generate a three-dimensional (3D) pleural surface observed from the inside of the lung and looking outward, and output the 3D pleural surface for display on the display device. In a first example of the system, the system also includes: an ultrasound probe configured to capture volumetric ultrasound data and / or a two-dimensional (2D) ultrasound image. In a second example of the system, optionally including the first example, the processor is further configured with instructions in the non-transitory memory, which, when executed, causes the processor to: construct a 3D mesh from a series of 2D images, and generate the 3D pleural surface from the 3D mesh. In a third example of the system, optionally including one or both of the first and second examples, the processor is further configured with instructions in the non-transitory memory, which, when executed, causes the processor to: generate the 3D pleural surface by stacking a series of 2D images. In a fourth example of the system, optionally including one or more or each of the first to third examples, the system further includes: a deep learning model trained for real-time and / or non-real-time pleural detection in volume data. In a fifth example of the system, optionally including one or more or each of the first to fourth examples, the processor is further configured with instructions in the non-transitory memory, which instructions, when executed, cause the processor to: apply at least one virtual light source to the 3D pleural surface by positioning the virtual light source within the inner side of the lung. In a sixth example of the system, optionally including one or more or each of the first to fifth examples, the system further includes: a user interface capable of being communicatively coupled to the processor and the display device, the user interface being configured to receive input for adjusting the position, color, brightness and / or other characteristics of the virtual light source.

[0098] The present disclosure also provides support for a method, the method comprising: acquiring volume ultrasound data; identifying a lung pleura in the volume ultrasound data; generating a three-dimensional (3D) pleural surface of the lung pleura viewed from the inside of the lung and looking outward based on the volume ultrasound data; removing the volume ultrasound data on one side of the lung pleura facing the inside of the lung; applying a light source to the 3D pleural surface from the inside; and outputting the 3D pleural surface to which the light source is applied for display. In a first example of the method, the 3D pleural surface is generated in real time when ultrasound image data is acquired. In a second example of the method, optionally including the first example, the 3D pleural surface is generated when ultrasound data acquisition is offline or frozen. In a third example of the method, optionally including one or both of the first and second examples, the method further comprises: selecting and outputting a representative 2D image of the 3D pleural surface, the representative 2D image including pleural irregularities also shown in the 3D pleural surface. In a fourth example of the method, optionally including one or more or each of the first to third examples, applying the light source includes positioning a virtual light source in the inner side of the lung at an angle relative to the 3D pleural surface.

[0099] As used herein, the elements or steps listed in the singular and beginning with the word "one" or "an" should be understood as not excluding a plurality of the elements or steps, unless such exclusion is explicitly stated. In addition, the reference to "one embodiment" of the present invention is not intended to be interpreted as excluding the existence of additional embodiments that also include the cited features. In addition, unless explicitly stated to the contrary, "comprising", "including" or "having" an embodiment of an element or multiple elements with a specific characteristic may include additional such elements that do not have the characteristic. The terms "including" and "in..." are used as the concise language equivalents of the corresponding terms "including" and "wherein". In addition, the terms "first", "second" and "third" etc. are only used as marks, and are not intended to impose numerical requirements or specific positional order on their objects.

[0100] The embodiments of the present disclosure shown in the drawings and described above are exemplary embodiments only and are not intended to limit the scope of the appended claims, including any equivalents included within the scope of the claims. Various modifications are possible and will be apparent to those skilled in the art. It is intended that any combination of non-mutually exclusive features described herein is within the scope of the invention. That is, the features of the embodiments may be combined with any appropriate aspect described above, and the optional features of any one aspect may be combined with any other appropriate aspect. Similarly, the features listed in the dependent claims may be combined with non-mutually exclusive features of other dependent claims, particularly where the dependent claims are dependent on the same independent claim. In some jurisdictions that require single claim dependencies, these dependencies may have been used as a practice, but this should not be taken to mean that the features in the dependent claims are mutually exclusive.

Claims

1. A method (300, 1100) for imaging a lung of a patient, comprising: A three-dimensional (3D) pleural surface viewed from the inside of the lung and looking outward is generated based on ultrasound imaging signals.

2. The method of claim 1, wherein the ultrasound imaging signals are captured by positioning a one-dimensional (1D), one and a half-dimensional (1.5D), or matrix (two-dimensional, 2D) probe (106) to acquire 2D images. 3 . The method of claim 1 , wherein the ultrasound imaging signals are captured by positioning a probe capable of live 3D imaging to acquire volumetric data.

4. The method according to claim 1, further comprising: generating an ultrasound image from the ultrasound imaging signal; detecting a bottom edge of the pleura in the ultrasound image; as well as Ultrasound data is removed from below the bottom edge of the pleura. 5 . The method of claim 1 , further comprising shading the 3D pleural surface via at least one virtual light source positioned at an acute angle relative to the 3D pleural surface as if inside the lung.

6. The method of claim 5, wherein the shading comprises: A pleural irregularity of the 3D pleural surface is highlighted, and the at least one virtual light source is positioned within the inner side of the lung at the acute angle relative to the pleural irregularity. The method of claim 5 , wherein a user controls the position of the at least one virtual light source via the display of the 3D pleural surface.

8. The method of claim 1, further comprising identifying and displaying B-lines of the 3D pleural surface.

9. A system (200), comprising: Display device (233); as well as A processor (231) configured with instructions in a non-transitory memory (206) that, when executed, cause the processor to: A three-dimensional (3D) pleural surface viewed from the inside of the lung and looking outward is generated and output for display on the display device. 10 . The system of claim 9 , further comprising an ultrasound probe configured to capture volumetric ultrasound data and / or two-dimensional (2D) ultrasound images.

11. The system of claim 9, wherein the processor is further configured with instructions in the non-transitory memory that, when executed, cause the processor to: constructing a 3D mesh from a series of 2D images; and The 3D pleural surface is generated from the 3D mesh.

12. The system of claim 9, wherein the processor is further configured with instructions in the non-transitory memory that, when executed, cause the processor to: The 3D pleural surface is generated by stacking a series of 2D images.

13. The system of claim 9, further comprising a deep learning model trained for real-time and / or non-real-time pleura detection in volumetric data.

14. The system of claim 9, wherein the processor is further configured with instructions in the non-transitory memory that, when executed, cause the processor to: At least one virtual light source is applied to the 3D pleural surface by positioning a virtual light source within the inner side of the lung.

15. The system of claim 14, further comprising a user interface communicatively coupled to the processor and the display device, the user interface configured to receive input for adjusting the position, color, brightness, and / or other characteristics of the virtual light source.

Images