Automated lung slide detection aids in the diagnosis of pneumothorax

Abstract

Methods and apparatus are disclosed for performing automated lung sliding detection using a computing device (e.g., an ultrasound system, etc.). In some embodiments, the technique uses one or more neural networks to determine lung sliding. In some embodiments, the neural network is part of a process that determines the probability of lung sliding in one or more M-lines. In some embodiments, the technique displays one or more probabilities of lung sliding in a B-mode ultrasound image.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Patent Application No. 17 / 734,586, filed May 2, 2022, which is incorporated by reference in its entirety.

[0002] The embodiments disclosed herein relate generally to ultrasound imaging, and more particularly, the embodiments disclosed herein relate to performing automated detection of lung slides using an ultrasound imaging system, including generating visualizations (e.g., three-dimensional images) indicative of the presence of lung sliding. [Background technology]

[0003] Lung ultrasound (US) represents a novel and promising approach to aid in the diagnosis of pneumothorax (PTX) with high sensitivity and specificity. Specifically, the diagnosis of PTX can be aided by the determination of lung sliding or non-sliding, and has been performed with ultrasound devices, which are determined using lung sliding / non-sliding metrics. Typically, these metrics involve movement relative to the pleural line in the ultrasound image. Currently, clinicians evaluate B-mode video clips for upward and downward movement of the pleural line. Clinicians also use M-mode to examine upward and downward movement of the pleural line. These techniques have disadvantages in that they must be performed by a skilled individual who is adept at recognizing lung sliding, and / or are time-consuming and prone to user error. These disadvantages may prevent these techniques from being used in real time in certain situations, thus affecting life-saving efforts. Summary of the Invention

[0004] Methods and apparatus for performing automated pulmonary slide detection using a computing device (e.g., an ultrasound system, etc.) are disclosed. In some embodiments, the methods are performed by a computing device.

[0005] In some embodiments, a method for determining lung sliding includes generating attribute quality probabilities for a B-mode ultrasound image including a pleural line and determining a quality level of the B-mode ultrasound image as acceptable for determining lung sliding based on the attribute quality probabilities. The method further includes generating one or more M-mode ultrasound images based on the B-mode ultrasound image and generating one or more probabilities of lung sliding based on the one or more M-mode ultrasound images.

[0006] In some embodiments, a method for determining lung sliding includes generating B-mode ultrasound images and generating an M-mode ultrasound image corresponding to an M-line. The method includes generating a probability of lung sliding at the M-line based on the M-mode ultrasound images and indicating the probability of lung sliding in at least one of the B-mode ultrasound images.

[0007] In some embodiments, a computer device implements an ultrasound system for determining lung sliding. In some embodiments, the computer device includes a memory that maintains a B-mode ultrasound image and one or more M-mode ultrasound images, and a neural network that is implemented at least in part in the hardware of the computer device and generates one or more probabilities of lung sliding at one or more M-lines based on the one or more M-mode ultrasound images. The computer device also includes a processor system that generates one or more M-mode ultrasound images corresponding to the one or more M-lines based on pixels in the B-mode ultrasound image corresponding to the one or more M-lines, and displays one or more representations of the one or more probabilities of lung sliding in at least one of the B-mode ultrasound images.

[0008] The present invention will be more fully understood from the following detailed description and the accompanying drawings of various embodiments of the invention, which should not be construed as limiting the invention to the specific embodiments, but are for illustration and understanding only. [Brief description of the drawings]

[0009] [Figure 1] 1A-1D illustrate several embodiments of an ultrasound machine. [Figure 2A] FIG. 1 is a diagram showing an example of a B-mode image. [Figure 2B] FIG. 1 is a diagram showing an example of a B-mode image. [Figure 3A] FIG. 1 shows an example of a good quality image. [Figure 3B] FIG. 1 shows an example of a low quality image. [Figure 4] FIG. 1 is a diagram showing an example of a pleural line. [Figure 5A] FIG. 2 shows an M-mode image constructed from a sequence of M lines of a B-mode video frame. [Figure 5B] FIG. 13 illustrates processing of M-mode images using a neural network to generate probabilities of lung sliding at three M-lines. [Figure 6] 1A-1C illustrate some embodiments of a system for performing lung sliding detection processing. [Figure 7] FIG. 13 is a data flow diagram of some embodiments of a lung sliding detection process. [Figure 8] FIG. 1 is a flow diagram of some embodiments of a process for generating an M-mode ultrasound image from a B-mode ultrasound image. [Figure 9A] 11 is a flow diagram of some embodiments of a lung sliding determination process. [Figure 9B] 13A-13C illustrate some embodiments of a pulmonary sliding determination process that generates an additional probability of pulmonary sliding to combine with other probabilities of pulmonary sliding. [Figure 10]11 is a flow diagram of some embodiments of another lung sliding determination process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] In the following description, numerous details are set forth in order to provide a thorough explanation of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the present invention.

[0011] Disclosed herein is a technique for automatically detecting lung sliding in ultrasound images generated using an ultrasound system. Lung sliding detection can be used to aid in the diagnosis of pneumothorax (PTX). US-based automated lung sliding detection can improve diagnostic accuracy and speed to reduce patient management time.

[0012] In some embodiments, the ultrasound system automatically detects lung sliding or non-sliding in ultrasound images through the use of one or more neural networks. These neural networks use models trained to determine lung sliding to help reduce inter-operator variability and implement consistent lung sliding detection algorithms. In some embodiments, the neural networks assist the user by acquiring video clips of acceptable quality to determine the presence of sliding in the lungs.

[0013] The ability to diagnose PTX in real time by automatically detecting lung sliding using a portable ultrasound device can be life-saving because the ultrasound device allows for the diagnosis of PTX at the point of care without the need to send the patient or images to a radiology department. Additionally, automatic lung sliding detection can improve diagnostic accuracy and speed, reducing patient management time.

[0014] The auto-detection algorithm and its implementation are described in more detail below.

[0015] Several embodiments of an ultrasound machine including embodiments of the disclosed technology are illustrated in Figure 1. Referring to Figure 1, an ultrasound transducer probe 100 includes an enclosure 110 extending between a distal end portion 112 and a proximal end portion 114. The ultrasound transducer probe 100 is electrically coupled to an ultrasound imaging system 130 via a cable 118 attached to the proximal end of the probe by a strain relief element 119. In some embodiments, the ultrasound transducer probe 100 is electrically coupled to the ultrasound imaging system 130 wirelessly.

[0016] A transducer assembly 120 having one or more transducer elements is electrically coupled to system electronics in the ultrasound imaging system 130. In operation, the transducer assembly 120 transmits ultrasonic energy from the one or more transducer elements towards a subject and receives ultrasonic echoes from the subject. The ultrasonic echoes are converted into electrical signals by the one or more transducer elements and electrically transmitted to system electronics in the ultrasound imaging system 130 to form one or more ultrasound images.

[0017] In general, capturing ultrasound data from a subject using an exemplary transducer assembly (e.g., transducer assembly 120) includes generating ultrasound waves, transmitting the ultrasound waves to the subject, and receiving ultrasound waves reflected by the subject. A wide range of ultrasound frequencies can be used to capture the ultrasound data, such as low frequency ultrasound (e.g., less than 15 MHz) and / or high frequency ultrasound (e.g., 15 MHz or greater). One of ordinary skill in the art can readily determine which frequency range to use based on factors such as, but not limited to, the imaging depth and / or the desired resolution.

[0018] In some embodiments, the ultrasound imaging system 130 includes ultrasound system electronics 134, including one or more processors, integrated circuits, ASICs, FPGAs, and power supplies, to support the functionality of the ultrasound imaging system 130 in a manner known in the art. In some embodiments, the ultrasound imaging system 130 also includes an ultrasound control subsystem 131 having one or more processors. At least one processor, FPGA, or ASIC causes the transducer(s) of the probe 100 to transmit electrical signals to emit sound waves, and receives electrical pulses from the probe resulting from returning echoes. The one or more processors, FPGAs, or ASICs process raw data associated with the received electrical pulses to form an image and transmit it to the ultrasound imaging subsystem 132, which displays the image on a display screen 133. The display screen 133 thus displays an ultrasound image from the ultrasound data processed by the processor of the ultrasound control subsystem 131.

[0019] In some embodiments, the ultrasound system may also have one or more user input devices (e.g., keyboard, cursor control device, microphone, camera, etc.) for inputting data to enable acquisition of measurements from the display of the ultrasound display subsystem, disk storage (e.g., hard disk, floppy disk, thumb drive, compact disk (CD), digital video disk (DVD)) for storing acquired images, and a printer for printing images from the displayed data. These devices are also not shown in FIG. 1 so as not to obscure the technology disclosed herein.

[0020] In some embodiments, the ultrasound system electronics 134 performs automatic lung sliding detection. Automatic detection of whether lung sliding is present can assist clinicians in diagnosing or ruling out pneumothorax, and includes benefits such as improved diagnostic accuracy and speed, reduced patient management time, and reduced inter-operator variability resulting from the use of a consistent lung sliding algorithm.

[0021] In some embodiments, automatic lung sliding detection is performed using an automated artificial intelligence (AI) algorithm that relies on the observation of multiple frames to determine whether sliding is present and its location in the body. In some embodiments, automatic detection is performed by sending a series of images to a neural network (e.g., a Convolutional Neural Network (CNN), a Swin Transformer, etc.). The series of images can be an ultrasound video clip, sent as either a group of stacked images into a single CNN, a series of images into a recurrent neural network (RNN), or a time-based AI model that can provide an indication (e.g., probability) of whether the images indicate the presence of lung sliding. The model can learn to detect sliding and its location in an image given appropriate training data with images fully annotated as to where sliding is present in each image. In some embodiments, the automatic detection process examines a single line of data as opposed to examining a frame as a whole. The single line of data can be an M-line from an M-mode image. These M-mode images can be generated in a number of ways. For example, an M-mode image may be acquired through an M-mode acquisition, which involves acquiring a single set of lines of data at a fixed rate (e.g., 100 lines per second) over a fixed time period (e.g., 1 second equals 100 lines of data). Additionally or alternatively, an M-mode image may be acquired by creating an M-mode image from a B-mode image.

[0022] In some embodiments, the automated detection process detects lung sliding from a single M-mode strip (hereinafter "M-strip") by creating one or more M-mode images based on one or more M-lines. That is, an M-strip is a series of three B-mode frames from which M-mode images are extracted at various M-lines. Details of these embodiments are described in more detail below. In some embodiments, the automated detection uses a neural network to examine a single M-strip to determine whether there is movement above and below the pleural line indicating that the lung is not collapsed. In some embodiments, if the acquisition frame rate is high enough, the automated detection process extracts multiple M-strips from a group of B-mode images (e.g., a two-dimensional (2-D) video clip, etc.) and uses a neural network to detect lung sliding from the M-strips. In some embodiments, the automated detection process extracts M-mode lines at a fixed angle to the vertical from each B-mode image, a technique often referred to as anatomical M-mode, and uses a neural network to examine these lines to determine whether lung sliding is present. In either of these cases, the neural network has a model trained using appropriate training data with images that are fully annotated as to where the sliding is present in each image, and learns to detect the sliding and its location in the input images.

[0023] 2A and 2B show an example of a B-mode image with a selected horizontal location (shown in the top center of the figure) and an M-strip at that location across multiple frames (shown below the B-mode image in the figure). In some embodiments, the M-strip is a three-dimensional data array (e.g., the x and y dimensions of the B-mode image and the z dimension of time, i.e., frames). In some embodiments, the M-strip is extracted from a series of B-mode images, and the M-mode images are reconstructed from a two-dimensional ultrasound video clip.

[0024] M-mode patterns that may indicate a lung with lung sliding (i.e., a lung that shows a normal pattern of aeration as the lung expands and contracts) include a horizontal line at the superficial pleural surface that is uninterrupted, and a granular pattern deep to this level. This is sometimes called the "seashore sign." FIG. 2A shows the "seashore sign" where there is a transition 203 between the "ocean" and the "beach" as indicated by the upward and downward movement of the pleural line 200 of the B-mode image 201, indicating that sliding is detected in the pleural line 200 of the M-mode image 202 (generated from multiple frames of the B-mode image 201). In contrast, FIG. 2B shows a pneumothorax (PTX) where there is a pattern sometimes called the "stratosphere" or "bar code" sign 213 in the M-mode image 212 (generated from multiple frames of the B-mode image 211), indicating that there is no movement in the pleural line 210 of the B-mode image 210, and thus no lung sliding.

[0025] Automatically detecting lung sliding by inspecting ultrasound images using neural networks has many advantages, including but not limited to its small computational requirements and ease of annotating the data (i.e., sliding or not sliding).

[0026] One challenge of the automated detection process using M-mode lines is determining which lines should be examined. In some embodiments, the determination as to which lines should be examined is made by first identifying and examining a region of interest (ROI) within the image from which an M-mode image is to be extracted (e.g., suitable for extracting M-lines). That is, the ROI indicates the set of M-lines from which a selection is made to extract an M-mode image. For example, when an M-line is selected at any M-line location (i.e., X image location) between the left and right portions of the ROI, an M-mode image is extracted from the M-strips at these M-line (i.e., X) locations. In some embodiments, as described above, this ROI spans the pleural line within the costal cavity of the lung. In one example, multiple M-lines from this region are examined to increase the accuracy of the sliding determination. It is also believed that different regions of the lung have different levels of sliding depending on the severity of the PTX observed.

[0027] Example of an Auto-Discovery Implementation In some embodiments, the automatic detection process has multiple processes, including determining image quality for lung sliding detection, determining ROI for lung sliding detection, determining acceptable image quality for M-mode reconstruction region, and determining lung sliding detection. Each of these operations will be described in detail below.

[0028] Image quality and region of interest (ROI) determination for lung sliding detection To ensure that lung sliding detection is evaluated on acceptable images, an AI model, referred to herein as a neural network (e.g., CNN, etc.), is trained to recognize images with acceptable image quality and suitable views for use in automatic lung sliding detection. In some embodiments, the determination of acceptable quality is based on one or more factors, including but not limited to, resolution, gain, brightness, clarity, centrality, depth, recognition of pleural lines, and recognition of ribs and / or rib shadows.

[0029] In some embodiments, the neural network recognizes the appropriate view by recognizing images with expected features such as the pleural line and ribs in the image. For example, in some embodiments, the neural network recognizes a clear pleural line in the top central region of the image and understands at least one rib shadow on one of the sides of the image. In one embodiment, the neural network is trained to recognize the location of the pleural line through different methods. These methods include, but are not limited to, the use of two points in the extension of the pleural line, left and right extent and center depth, segmentation maps, and heat maps.

[0030] In some embodiments, the data output from the neural network can be used in combination with heuristics to determine acceptable or good quality images, or unacceptable or poor quality images. An example of a good quality image is shown in FIG. 3A. The neural network can determine that an image is of poor quality and unacceptable because of one or more attributes, such as the image being too dark, too light, too blurry, too deep, too shallow, off-center, etc., in addition to not recognizing expected features such as the pleural line and ribs, and can determine that an image is of good quality and acceptable when it does not have these attributes that make it unacceptable. An example of a good quality image is shown in FIG. 3A. An example of a poor quality image is shown in FIG. 3B. In some embodiments, the neural network outputs the indications of good and poor quality as good / bad probabilities of multiple attributes.

[0031] In addition to calculating the good / bad probabilities of multiple attributes, the neural network can also detect the locations (e.g., x,y positions or coordinates) of two points that indicate the left and right edges or end points of the pleural line in the image. An example of a pleural line is shown in Figure 4. In Figure 4, markers 401 and 402 indicate the end points of the pleural line.

[0032] In some embodiments, the good / bad probability generated by the neural network is used in combination with a heuristic rule that uses the x / y location of the pleural line to determine the overall quality of the B-mode ultrasound image. In some embodiments, the x location is used to determine whether the pleural line spans a prescribed distance in the image. In some embodiments, the prescribed distance is based on a percentage of the image centered on the image's center point. For example, a line segment formed by connecting the ROI points must cross the center of the image. If the pleural line does not span the prescribed distance, the image is deemed bad. The y location of the pleural line can be used to determine if the image is too deep or too shallow. This location information can be used to determine a region of interest (ROI) for computing metrics of lung sliding. For example, the x location of the pleural line from the model can be used to determine an ROI that can be used to select M-line locations for a reconstructed M-mode image.

[0033] Determining the acceptable quality of M-mode reconstruction regions (M-strips) M-mode images can be reconstructed from the M-strips. Before reconstructing an M-mode image from the M-strips, the frames are inspected to determine whether the M-strips are acceptable for lung sliding determination. This determination can be based on the reported quality of each frame in the M-strip being good. Additionally or alternatively, in some embodiments, the lung sliding detection process checks the ROI points to determine whether there is too much motion. If there is too much motion, it may be difficult to determine the presence or absence of lung sliding in the reconstructed M-mode. By looking for excessive motion, the M-strips are marked as good or poor quality. If the quality is poor, the M-strips are not used for lung sliding detection. In some embodiments, to detect motion in the M-strip frames, the change in the x,y position of the pleural line in successive B-mode frames can be compared to each other to see if it exceeds a prescribed limit. If the change in the x,y position of the pleural line exceeds a prescribed limit, the motion of the M-strip frame is too large to be used to determine the presence or absence of lung sliding. However, the neural network can determine whether the overall motion in the B-mode image is too large to be used for lung sliding detection. For example, the neural network can examine the ROIs on every frame, and if there is a misalignment of the points throughout the frames, it determines that the M-mode image reconstructed from the B-mode image is not of good enough quality.

[0034] Once an M-strip is designated as good quality, an M-mode image can be reconstructed for any M-line of the B-mode image within the ROI. In some embodiments, an M-mode image can be reconstructed by selecting a given M-line column of vertical image pixels from each frame (e.g., 25 frames) within the M-strip. This process can be repeated for all selected frames. Combining these vertical columns produces an M-mode image with a pulse repetition frequency (PRF) equal to the frame rate of the video clip.

[0035] FIG. 5A shows an M-mode image constructed from an M-line sequence of B-mode video frames. FIG. 5A shows B-mode video frames forming an M-strip 501 (e.g., 25 frames, etc.) with the M-line sequence 502 highlighted. The same sequences of the M-line sequence 502 in each of the B-mode video frames 501 are combined to create an M-mode image 503. Although only three M-mode images 503 are shown in FIG. 5A, fewer or more than three M-mode images 503 may be constructed from the M-line sequence 502 of the B-mode video frames 501. It should be noted that lung sliding can be performed by evaluating multiple M-mode images constructed in this manner. For example, a window that is three or more pixels wide can be examined as a region of interest in the M-mode image 509. This window can be a sliding window that is examined to determine whether lung sliding is present anywhere in the region.

[0036] Alternatively or in addition to constructing an M-mode image as described above, a lung sliding detection process can be performed on stored images (e.g., a CINE loop having a series of digital images from an ultrasound examination) to detect lung sliding.

[0037] Assessment of lung sliding In some embodiments, a second neural network is trained to distinguish between M-mode images that show lung sliding and M-mode images that show the absence of lung sliding. The reconstructed M-mode images are fed into this model to determine the presence or absence of sliding. In some embodiments, this determination is based on only one M-mode image. In some embodiments, this determination is based on multiple M-mode images. For example, the ultrasound system can construct a variable number of M-mode images depending on available computational resources and response time, and pass them through the lung sliding model to determine the presence or absence of sliding. This detection can be done for multiple M-mode images constructed from different M-line positions in the M-strip. This detection can also be done for multiple M-strips (e.g., different series of B-mode images that may or may not be consecutive in time). All lung sliding detection outputs can be combined to obtain a higher average accuracy than when looking at lung sliding model detection from a single reconstructed M-mode. In some embodiments, the lung sliding detection outputs are combined using an average function to achieve high accuracy.

[0038] FIG. 5B illustrates the processing of an M-mode image using a neural network to generate a probability of lung sliding at three M-lines. With reference to FIG. 5B, an M-mode image 510 is input to the neural network 510 to generate a B-mode image 511 with an M-line 512 between pleural line endpoints 521 and 522. The M-mode image 510 is an example of an M-mode image generated from an M-strip of a B-mode image, such as the M-mode image 503 of FIG. 5A, and the M-line 512 is an example of an M-line selected from an M-strip, such as the M-line column 502 of the M-strip 501. Although FIG. 5B clearly shows three mode images 510 being input to the neural network 510, in alternative embodiments, more or less than three M-mode images may be input to the neural network to detect lung sliding.

[0039] In some embodiments, M-lines 512 are displayed in the B-mode image 511 with an indication of the probability of lung sliding. For example, one of the M-lines 512 can be a particular shade of color (e.g., green) indicating sliding, and another one of the M-lines 511 can be displayed on the B-mode image 511 with a shade of color (e.g., red) indicating low or zero probability of lung sliding. In this example shown in FIG. 5B, the number of M-lines 512 displayed is equal to three. However, the techniques described herein are not limited to displaying only three M-lines. It is noted that there may be M-lines 512 for every line in the M-mode image 510. In such an example, these lines may indicate the beginning of lung sliding into an area where there is no lung sliding. Additionally or alternatively, the user may select which M-lines to display in the B-mode image 511.

[0040] Example of a lung sliding detection system Some embodiments of a system for performing lung sliding detection processing are shown in FIG. 6. Referring to FIG. 6, a B-mode image 601 is provided to a quality check neural network (model) 602 and a region of interest neural network (model) 603. In one embodiment, the quality check neural network (model) 602 and the region of interest neural network (model) 603 are separate neural networks. In some embodiments, these neural networks are combined into one neural network. In yet other embodiments, these networks share at least one common portion and include other portions that are not shared between these networks.

[0041] A quality check neural network 602 receives the B-mode images 601 and determines whether each of the B-mode images 601 is of sufficient quality to be used in the lung sliding detection process. The quality check neural network 602 determines the quality as described above and outputs a quality level indication 610 for each of the B-mode images. In some embodiments, the quality is output and displayed on a display screen (e.g., a display screen of an ultrasound machine, etc.) to allow a user to guide and improve their image acquisition.

[0042] The region of interest neural network 603 receives the B-mode images 601 and determines the location of the pleural line 611. The ROI neural network 603 outputs location information 611 for each of the B-mode images. In some embodiments, the location information includes a set of coordinates of the endpoints of the pleural line. In some embodiments, these coordinates are the x,y coordinates of the endpoints of the pleural line in each of the B-mode images.

[0043] The quality level indication 610 and the position information 611 are input to the M mode image generator 604 along with the B mode image 601. In response to these inputs, the M mode image generator 604 generates a reconstructed M mode image 612. As mentioned above, in some embodiments, the M mode image generator 604 generates the reconstructed M mode image 612 from the B mode image. Additionally or alternatively, the M mode image can be acquired through a well-known M mode image acquisition process.

[0044] A lung sliding detection neural network (model) 605 receives the reconstructed M-mode images 612 and performs lung sliding detection on the reconstructed M-mode images 612. In some embodiments, lung sliding detection is performed as described above. The lung sliding detection neural network 605 generates as output lung sliding detection results 613. In some embodiments, the lung sliding detection results 613 include a probability of lung sliding associated with each image. As described above, the lung sliding detection results can be displayed on the ultrasound image, such as a B-mode image. For example, the ultrasound system can display the lung sliding detection results as part of a heat bar as described above and / or as part of a binary icon distinguishing between the presence or absence of lung sliding, such as a thumbs up / thumbs down indicator.

[0045] One or more of the neural networks of FIG. 6 can be implemented in many different ways. In one embodiment, the neural network includes a model using a sequence model including an EfficientNet architecture, a convolutional neural network (CNN), and / or a recurrent neural network (RNN). It is noted that the detection techniques described herein can be implemented with artificial intelligence (AI) or machine learning (e.g., adaptive boosting (adaboost), deep learning, supervised learning models, support vector machines (SVM), gated recurrent units (GRU), convolutional GRU (ConvGRU), long short-term memory (LSTM), etc., that process frame information in sequence), and / or another suitable detection method.

[0046] Example flow diagram of lung detection process 7 is a data flow diagram of some embodiments of a lung sliding detection process. The process can be performed by processing logic that can include hardware (e.g., circuitry, dedicated logic, memory, etc.), software (such as running on a general-purpose computer system or a dedicated machine), firmware (e.g., software programmed in a read-only memory), or a combination thereof. In some embodiments, the process is performed by one or more processors of a computing device, such as, but not limited to, an ultrasound machine including an ultrasound imaging subsystem.

[0047] 7, the process begins by processing logic (e.g., one or more memories) generating a B-mode ultrasound image (processing block 701). Processing logic generates one or more M-mode ultrasound images corresponding to the one or more M-lines (processing block 702). In some embodiments, the one or more M-mode images are generated based on the pixels of the B-mode image and the one or more M-lines.

[0048] Processing logic generates one or more probabilities of lung sliding in one or more M-lines based on the one or more M-mode ultrasound images (processing block 703). In one embodiment, processing logic generates one or more probabilities of lung sliding in one or more M-lines using a neural network. In some embodiments, the neural network is implemented at least in part in the hardware of the computing device.

[0049] After generating one or more probabilities of lung sliding in one or more M-lines, processing logic displays or otherwise illustrates a representation of the probabilities of lung sliding in at least one B-mode ultrasound image (processing block 704).

[0050] 8 is a flow diagram of some embodiments of a process for generating an M-mode ultrasound image from a B-mode ultrasound image. The process may be performed by processing logic that may include hardware (e.g., circuitry, dedicated logic, memory, etc.), software (such as executed on a general-purpose computer system or a dedicated machine), firmware (e.g., software programmed into a read-only memory), or a combination thereof. In some embodiments, the process is performed by one or more processors of a computing device, such as, for example, but not limited to, an ultrasound machine that includes an ultrasound imaging subsystem.

[0051] With reference to FIG. 8, the process begins by processing logic generating a quality level for a B-mode ultrasound image (processing block 801) and determining whether the quality level for the B-mode ultrasound image is greater than a quality threshold (processing block 802). In one embodiment, processing logic generates the quality level for the B-mode ultrasound image based on attribute quality probabilities and coordinate pairs. Examples of attribute quality probabilities include probabilities of attribute qualities including resolution, gain, brightness, clarity, centrality, depth, pleural line recognition, and rib recognition. In some embodiments, processing logic generates attribute quality probabilities for the B-mode image and coordinate pairs indicative of edges (e.g., endpoints) of the pleural line in the B-mode image. In some embodiments, processing logic generates the attribute quality probabilities using a neural network. In some embodiments, the neural network is implemented at least partially within the hardware of the computing device.

[0052] After processing logic determines whether the quality level is greater than the quality threshold, it generates one or more M-mode ultrasound images (processing block 803). In some embodiments, processing logic generates one or more M-mode ultrasound images in response to the quality level being greater than the quality threshold. In other words, an M-mode ultrasound image is generated only if the quality of the B-mode image is greater than the quality threshold.

[0053] Then, processing logic generates one or more probabilities of lung sliding in one or more M lines of the M mode ultrasound image (processing block 804). In some embodiments, the one or more probabilities are based on the M mode ultrasound image generated from the B mode ultrasound image.

[0054] 9A is a flow diagram of some embodiments of a lung sliding determination process. The process can be performed by processing logic that can include hardware (e.g., circuitry, dedicated logic, memory, etc.), software (such as running on a general-purpose computer system or a dedicated machine), firmware (e.g., software programmed in a read-only memory), or a combination thereof. In some embodiments, the process is performed by one or more processors of a computing device, such as, but not limited to, an ultrasound machine that includes an ultrasound imaging subsystem.

[0055] 9A, the process begins by generating attribute quality probabilities for a B-mode ultrasound image and coordinate pairs indicative of edges of the pleural line in the B-mode ultrasound image (processing block 901). In some embodiments, the attribute quality probabilities for these B-mode ultrasound images and coordinate pairs are generated using a first neural network implemented at least in part in the hardware of a computing device.

[0056] Next, processing logic determines a region of interest within the B-mode ultrasound image (processing block 902). In some embodiments, the region of interest within the B-mode ultrasound image is determined based on a previously generated coordinate pair.

[0057] Processing logic also determines a quality level of the B-mode ultrasound image as acceptable for lung sliding determination (processing block 903). In some embodiments, the determination that the B-mode ultrasound image has an acceptable quality level for lung sliding determination is made based on the previously generated attribute quality probabilities and the amount of motion in the region of interest. In some embodiments, the attribute quality probabilities indicate a probability of at least one attribute quality selected from the group consisting of resolution, gain, brightness, clarity, centrality, depth, pleural line recognition, and rib recognition.

[0058] In some embodiments, determining the quality level as acceptable includes determining a horizontal span of the pleural line for each B-mode ultrasound image and comparing the horizontal span to a threshold distance. In some embodiments, determining the horizontal span of the pleural line is performed based on a horizontal component of a coordinate pair. In some embodiments, the process includes processing logic that sets the threshold distance to be a percentage of the size of at least one B-mode ultrasound image. For example, in some embodiments, the pleural line must be located vertically across 20%-60% of the image to be considered good quality, and the pleural line must cross the center of the image. In some embodiments, determining the quality level as acceptable includes determining, for each B-mode ultrasound image, a depth of each B-mode ultrasound image based on a vertical component of a coordinate pair.

[0059] Processing logic uses the B-mode ultrasound images to generate one or more M-mode ultrasound images corresponding to one or more M-lines in the region of interest (processing block 904). In some embodiments, the M-mode ultrasound images are from pixel columns in each B-mode ultrasound image that correspond to one or more M-lines.

[0060] Processing logic generates a probability of lung sliding in one or more M-lines based on one or more M-mode ultrasound images (processing block 905). In some embodiments, processing logic generates a probability of lung sliding in one or more M-lines using a neural network. The neural network may be implemented at least in part in hardware of a computing device (e.g., an ultrasound machine such as ultrasound system 130 of FIG. 1).

[0061] Processing logic may also display a visual representation of one or more M-lines indicating a probability of lung sliding in the one or more M-lines (processing block 906). The color-coded version of M-lines 512 shown in FIG. 5B is an example of a visual representation of one or more M-lines indicating the probability with color. In some embodiments, processing logic displays the representation of these M-lines in a B-mode ultrasound image. In some embodiments, processing logic displays the visual representation horizontally across the region of interest, such as via a heat bar as described above. In some embodiments, the process of generating the visual representation includes processing logic filtering the probabilities. The probabilities may be filtered horizontally using a smoothing function.

[0062] In some embodiments, the one or more M-mode ultrasound images include a plurality of M-mode ultrasound images, and the one or more M-lines include a plurality of M-lines across the region of interest, in some embodiments in such cases, the process generates a visual representation of the probability of lung sliding at the plurality of M-lines and displays the visual representation horizontally across the region of interest.

[0063] In such a case, the processing logic may generate a plurality of M-mode ultrasound images based on the first starting frame of the B-mode ultrasound images. In some embodiments, the process includes generating a further M-mode ultrasound image based on a second starting frame of the B-mode ultrasound images to generate further probabilities of lung sliding in the plurality of M-lines. The process also includes combining the probabilities and the further probabilities to form combined probabilities of lung sliding in the plurality of M-lines. After forming the combined probabilities, the process generates and displays a visual representation of the combined probabilities. In some embodiments, the processing logic uses a neural network to generate further probabilities of lung sliding in the plurality of M-lines based on the further M-mode ultrasound images.

[0064] 9B shows some embodiments of a lung sliding determination process that generates a further probability of lung sliding and combines it with other probabilities of lung sliding. The process can be performed by processing logic that can include hardware (e.g., circuitry, dedicated logic, memory, etc.), software (such as running on a general-purpose computer system or a dedicated machine), firmware (e.g., software programmed in a read-only memory), or a combination thereof. In some embodiments, the process is performed by one or more processors of a computer device, such as, but not limited to, an ultrasound machine that includes an ultrasound imaging subsystem.

[0065] 9B, the process begins by generating attribute quality probabilities for a B-mode ultrasound image and coordinate pairs indicative of edges of the pleural line in the B-mode ultrasound image (processing block 911). In some embodiments, the attribute quality probabilities for these B-mode ultrasound images and coordinate pairs are generated using a first neural network implemented at least in part in the hardware of a computing device.

[0066] Next, processing logic determines a region of interest within the B-mode ultrasound image (processing block 912). In some embodiments, the region of interest within the B-mode ultrasound image is determined based on the previously generated coordinate pairs.

[0067] Processing logic also determines a quality level of the B-mode ultrasound image as acceptable for lung sliding determination (processing block 913). In some embodiments, the determination that the B-mode ultrasound image has an acceptable quality level for lung sliding determination is made based on the previously generated attribute quality probabilities and the amount of motion in the region of interest. In some embodiments, the attribute quality probabilities indicate a probability of at least one attribute quality selected from the group consisting of resolution, gain, brightness, clarity, centrality, depth, pleural line recognition, and rib recognition.

[0068] In some embodiments, determining the quality level as acceptable includes determining a horizontal span of the pleural line for each B-mode ultrasound image and comparing the horizontal span to a threshold distance. In some embodiments, determining the horizontal span of the pleural line is performed based on a horizontal component of the coordinate pair. In some embodiments, the process includes processing logic that sets the threshold distance to be a percentage of the size of at least one B-mode ultrasound image. For example, in some embodiments, the pleural line must be located vertically across 20%-60% of the image to be considered good quality, and the pleural line must cross the center of the image. In some embodiments, determining the quality level as acceptable includes determining, for each B-mode ultrasound image, a depth for each of these B-mode ultrasound images based on a vertical component of the coordinate pair.

[0069] Processing logic uses the B-mode ultrasound images to generate one or more M-mode ultrasound images corresponding to one or more M-lines in the region of interest (processing block 914). In some embodiments, the M-mode ultrasound images are from pixel columns in each B-mode ultrasound image that correspond to one or more M-lines.

[0070] Processing logic generates a probability of lung sliding based on one or more M-mode images (e.g., at one or more M-lines) based on one or more M-mode ultrasound images (processing block 915). In some embodiments, processing logic generates the probability of lung sliding at one or more M-lines using a neural network. The neural network may be implemented at least in part in the hardware of a computing device (e.g., an ultrasound machine such as ultrasound system 130 of FIG. 1).

[0071] Next, processing logic generates an additional M-mode ultrasound image based on the second starting frame of B-mode ultrasound images (processing block 916), and generates an additional probability of lung sliding based on the additional M-mode ultrasound image (processing block 917), which in some embodiments are generated in the same manner as described above in connection with processing blocks 914 and 915.

[0072] Processing logic combines the multiple probabilities generated from processing block 915 with additional probabilities to form a combined probability of lung sliding (processing block 916).

[0073] Processing logic may also generate a visual representation of the combined probabilities (processing block 919) and display the visual representation (processing block 920). The color-coded version of M-lines 512 shown in FIG. 5B is an example of a visual representation of one or more M-lines that indicates probabilities with color. In some embodiments, processing logic displays the representation of these M-lines within a B-mode ultrasound image. In some embodiments, processing logic displays the visual representation horizontally across the region of interest, such as via a heat bar as described above. In some embodiments, the process of generating the visual representation includes processing logic filtering the probabilities. The probabilities may be filtered horizontally using a smoothing function.

[0074] 10 is a flow diagram of some embodiments of another lung sliding determination process. The process can be performed by processing logic that can include hardware (e.g., circuitry, dedicated logic, memory, etc.), software (such as running on a general-purpose computer system or a dedicated machine), firmware (e.g., software programmed in a read-only memory), or a combination thereof. In some embodiments, the process is performed by one or more processors of a computing device, such as, but not limited to, an ultrasound machine that includes an ultrasound imaging subsystem.

[0075] 10, the process begins by generating a B-mode ultrasound image (processing block 1001). In some embodiments, the B-mode ultrasound image is generated in a manner known in the art.

[0076] In some embodiments, processing logic determines a quality level of the B-mode ultrasound image (processing block 1002). In some embodiments, processing logic determines the quality level using a process that includes generating coordinate pairs indicative of edges of a pleural line in the B-mode ultrasound image, determining a region of interest in the B-mode ultrasound image based on the coordinate pairs, and determining an amount of motion in the region of interest. In some embodiments, the coordinate pairs indicative of edges of the pleural line in the B-mode ultrasound image are generated using a neural network. This neural network can be in addition to the neural network that generates the probability of lung sliding at the M-lines. In some embodiments, the neural network that generates the coordinate pairs is implemented at least partially within the hardware of the ultrasound system.

[0077] In some embodiments, the processing logic determines the quality level using a process that includes using a further neural network implemented at least in part in the hardware of the ultrasound system to generate coordinate pairs indicative of edges of the pleural line in the B-mode ultrasound image. The quality level determination process can include determining a horizontal span of the pleural line based on the coordinate pairs and comparing the horizontal span to a threshold distance. In some embodiments, the processing logic uses a neural network to generate coordinate pairs indicative of edges of the pleural line in the B-mode ultrasound image. This neural network can be in addition to the neural network that generates the probability of lung sliding at the M-line. In some embodiments, the neural network that generates the coordinate pairs is implemented at least in part in the hardware of the ultrasound system.

[0078] In some embodiments, the processing logic determines the quality level using a process that includes generating attribute quality probabilities for the B-mode ultrasound image indicative of a probability of at least one attribute quality selected from the group consisting of resolution, gain, brightness, clarity, centrality, depth, pleural line recognition, and rib recognition. In some embodiments, the processing logic generates the attribute quality probabilities for the B-mode ultrasound image using a neural network. This neural network can be in addition to the neural network that generates the probability of lung sliding at M-lines. In some embodiments, the neural network that generates the attribute quality probabilities is implemented at least in part within the hardware of the ultrasound system.

[0079] Processing logic then discards a first portion of the B-mode ultrasound image based on the quality levels of the B-mode ultrasound image (processing block 1003), while retaining a second portion of the B-mode ultrasound image based on the quality levels (processing block 1004). In some embodiments, a probability of lung sliding is based on the retained portions of the B-mode ultrasound image. Also, in some embodiments, the quality may be displayed to a user.

[0080] Processing logic uses the retained B-mode ultrasound image to generate an M-mode ultrasound image corresponding to the M-line (processing block 1005). Note that this process may be repeated to generate multiple M-mode ultrasound images. In some embodiments, processing logic generates the M-mode image based on pixels in the B-mode ultrasound image that correspond to the M-line.

[0081] Processing logic generates a probability of lung sliding at each M-line based on the M-mode ultrasound image (processing block 1006). In some embodiments, processing logic generates the probability of lung sliding at the M-line using a neural network. In some embodiments, the neural network is implemented at least in part within the hardware of the ultrasound system.

[0082] After generating the probability of lung sliding at the M-line, processing logic generates a further B-mode ultrasound image (processing block 1007) and indicates the probability of lung sliding in the further B-mode ultrasound image (processing block 1008).

[0083] The systems, devices, and methods disclosed herein provide numerous advantages over conventional ultrasound systems, devices, and methods that aid in the diagnosis of PTX without implementing automated lung slide detection. For example, the ultrasound system disclosed herein can reliably diagnose PTX in real time using a portable ultrasound device, a diagnosis that simply cannot be made with conventional ultrasound systems due to the time required to operate the conventional ultrasound system and the errors introduced by the operator. As a result, the ultrasound system can diagnose PTX more accurately and quickly than conventional ultrasound systems, which can have a life-saving effect in the medical field.

[0084] Furthermore, the use of the ultrasound system disclosed herein reduces the resource burden of the care facility compared to the use of conventional ultrasound systems, because the ultrasound system disclosed herein allows the diagnosis of PTX to be successfully performed using only the ultrasound system, without the need to send the patient to a separate imaging department, such as radiology. In contrast, conventional ultrasound systems, as described above, may not be able to adequately diagnose PTX and may require the use of additional imaging, thus placing a higher burden on the resources of the care facility than the ultrasound system disclosed herein. Thus, the ultrasound system disclosed herein allows the care facility to operate more efficiently and thus provide better patient care, compared to conventional ultrasound systems.

[0085] Furthermore, since the ultrasound system disclosed herein operates faster than conventional ultrasound systems that do not implement automatic lung slide detection to aid in the diagnosis of PTX, an operator can use the ultrasound system disclosed herein to perform a more extensive ultrasound examination in a given time period than conventional ultrasound systems. Thus, the ultrasound system disclosed herein can provide a better patient care than conventional ultrasound systems. Throughout this specification and claims, words such as "comprise, " "comprising" and the like should be construed in an inclusive sense, i.e., "including, but not limited to," as opposed to an exclusive or exhaustive sense, unless the context clearly requires otherwise. As used herein, the terms "connected," "coupled," or any variation thereof, refer to a direct or indirect connection or coupling between two or more elements, and the connection or coupling between the elements can be physical, logical, or a combination thereof. Additionally, the words "herein," "above," "below," and words of similar import, when used in this application, refer to this application as a whole and not to any particular portions of this application. Where appropriate, words using the singular or plural forms in the above Detailed Description may also include the plural or singular, respectively. The word "or" in reference to a list of two or more items covers all interpretations of the word: any of the items in the list, all of the items in the list, and any combination of the items in the list.

[0086] Numerous variations and modifications of the present invention will become apparent to those skilled in the art upon reading the above description, but it should be understood that any particular embodiments shown and described by way of example are not intended to be considered limiting in any way. Accordingly, references to details of various embodiments are not intended to limit the scope of the claims, which recite only those features regarded as essential to the invention. [Explanation of symbols]

[0087] 100 Ultrasonic transducer probe 110 Enclosure 112 Distal end section 114 Proximal end portion 118 Cable 119 Strain Relief Elements 120 Transducer Assembly 130 Ultrasound System 131 Ultrasonic Control Subsystem 132 Ultrasound Imaging Subsystem 133 Display Screen 134 Ultrasound System Electronics

Claims

1. 1. A method for determining lung sliding, the method comprising the steps of: generating attribute quality probabilities for a B-mode ultrasound image including a pleural line; determining a quality level of the B-mode ultrasound image as acceptable for determining lung sliding based on the attribute quality probabilities; generating one or more M-mode ultrasound images based on the B-mode ultrasound images; generating one or more probabilities of lung sliding based on the one or more M-mode ultrasound images; The method according to claim 1, further comprising:

2. the attribute quality probability indicates a probability of at least one attribute quality selected from the group consisting of resolution, gain, brightness, clarity, centrality, depth, pleural line recognition, and rib recognition; The method of claim 1.

3. generating the attribute quality probabilities includes generating the attribute quality probabilities using a first neural network implemented at least in part in the hardware of the computer device, and generating the one or more probabilities of lung sliding includes generating the one or more probabilities of lung sliding using a second neural network implemented at least in part in the hardware of the computer device. The method of claim 1.

4. generating coordinates indicative of end points of the pleural line; determining a region of interest in the B-mode ultrasound image based on the coordinates; determining an amount of motion in the region of interest; determining the quality level as acceptable is based on the amount of motion; The method of claim 1.

5. generating coordinates indicative of end points of the pleural line; determining a horizontal span of the pleural line based on the coordinates; comparing the horizontal span to a threshold distance; determining the quality level as acceptable is based on the comparing. The method of claim 1.

6. and setting the threshold distance as a percentage of a size of at least one of the B-mode ultrasound images. The method according to claim 5.

7. generating coordinates indicative of end points of the pleural line; determining that the quality level is acceptable includes, for each of the B-mode ultrasound images, determining a depth of each of the B-mode ultrasound images based on the coordinates. The method of claim 1.

8. extracting one or more pixel rows corresponding to one or more M-lines in each of the B-mode ultrasound images; generating the one or more M-mode ultrasound images based on the one or more pixel columns, and the one or more probabilities of lung sliding correspond to the one or more M-lines. The method of claim 1.

9. generating one or more visual representations of the one or more M-lines, each indicative of the one or more probabilities of pulmonary sliding in the one or more M-lines; displaying the one or more visual representations within at least one of the B-mode ultrasound images; and The method of claim 8 further comprising:

10. and wherein the one or more M-mode ultrasound images include a plurality of M-mode ultrasound images, and the one or more probabilities include a plurality of probabilities, and generating the plurality of M-mode ultrasound images is based on a first starting frame of the B-mode ultrasound image, the method comprising: generating a further M-mode ultrasound image based on a second starting frame of the B-mode ultrasound image; generating a further probability of the lung sliding based on the further M-mode ultrasound image; combining the plurality of probabilities with the further probability to form a combined probability of the lung sliding; generating a visual representation of the combined probabilities; and displaying the visual representation; and The method of claim 1 further comprising:

11. The one or more M-mode ultrasound images include a plurality of M-mode ultrasound images corresponding to a plurality of M-lines intersecting a region of interest in the B-mode ultrasound image, and the one or more probabilities include a plurality of probabilities of the lung sliding in the plurality of M-lines, and the method further comprises: generating a visual representation of the plurality of probabilities of the lung sliding at the plurality of M-lines; displaying the visual representation horizontally across the region of interest; The method of claim 1 further comprising:

12. generating the visual representation includes horizontally filtering the plurality of probabilities using a smoothing function; The method of claim 11.

13. 1. A method for determining lung sliding, the method comprising the steps of: generating a B-mode ultrasound image; generating an M-mode ultrasound image corresponding to the M-line; generating a probability of the lung sliding at the M-line based on the M-mode ultrasound image; indicating the probability of the lung sliding in at least one of the B-mode ultrasound images; The method according to claim 1, further comprising:

14. generating the M-mode ultrasound image based on pixels in the B-mode ultrasound image corresponding to the M-lines; The method of claim 13.

15. determining a quality level of the B-mode ultrasound image; discarding a first portion of the B-mode ultrasound image based on the quality level of the B-mode ultrasound image in the first portion; retaining a second portion of the B-mode ultrasound image based on the quality level of the B-mode ultrasound image in the second portion; and generating the probability of lung sliding based on the second portion of the B-mode ultrasound image. The method of claim 13.

16. generating a visual representation of the quality level of the B-mode ultrasound image; displaying the visual representation within at least one of the B-mode ultrasound images to guide placement of an ultrasound probe; The method of claim 15.

17. Determining the quality level includes: generating coordinates indicating end points of a pleural line in the B-mode ultrasound image; determining a region of interest in the B-mode ultrasound image based on the coordinates; determining an amount of motion in the region of interest; 16. The method of claim 15, comprising:

18. Determining the quality level includes: generating coordinates indicating end points of a pleural line in the B-mode ultrasound image; determining a horizontal span of the pleural line based on the coordinates; comparing the horizontal span to a threshold distance; 16. The method of claim 15, comprising:

19. 1. A computing device implementing an ultrasound system for determining lung sliding, comprising: a memory for maintaining a B-mode ultrasound image and one or more M-mode ultrasound images; a neural network implemented at least in part in the hardware of the computing device to generate one or more probabilities of the lung sliding in one or more M-lines based on the one or more M-mode ultrasound images; generating the one or more M-mode ultrasound images corresponding to the one or more M-lines based on pixels in the B-mode ultrasound image corresponding to the one or more M-lines; displaying one or more representations of the one or more probabilities of lung sliding within at least one of the B-mode ultrasound images. A processor system; A computer device comprising:

20. The processor system includes: generating a quality level for the B-mode ultrasound image; determining whether the quality level is greater than a quality threshold; generating the one or more M-mode ultrasound images in response to the quality level being greater than the quality threshold.

20. The computer device of claim 19 implemented as follows:

Images