Automated lung slide detection useful for diagnosing pneumothorax

Abstract

Methods and apparatus are disclosed for performing automated lung sliding detection using a computing device (e.g., an ultrasound system, etc.). In some embodiments, the technique uses one or more neural networks to determine lung sliding. In some embodiments, the neural network is part of a process that determines the probability of lung sliding in one or more M-lines. In some embodiments, the technique displays one or more probabilities of lung sliding in a B-mode ultrasound image.

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application claims the benefit of U.S. Patent Application No. 17 / 734,586, filed May 2, 2022, which is hereby incorporated by reference in its entirety.

[0002] Embodiments disclosed herein generally relate to ultrasound imaging. Specifically, embodiments disclosed herein relate to performing automatic detection of lung slide using an ultrasound imaging system, including generating visualizations (e.g., 3D images) indicating the presence of lung sliding.

Background Art

[0003] Lung ultrasound (US) represents a new and promising approach for diagnosing pneumothorax (PTX) with high sensitivity and specificity. Specifically, in the diagnosis of PTX, determination of lung sliding or non - sliding can be useful, and diagnosis of PTX using an ultrasound device has been performed, which is determined using lung sliding / non - sliding metrics. Usually, these metrics involve movement related to the pleural line in an ultrasound image. Currently, clinicians evaluate B - mode video clips for movement above and below the pleural line. Also, clinicians use M - mode to examine movement above and below the pleural line. These techniques have disadvantages in that they must be performed by skilled practitioners proficient in recognizing lung sliding and / or are time - consuming and prone to user error. These disadvantages prevent these techniques from being used in real - time in certain situations, which may affect life - saving activities.

Summary of the Invention

[0004] Disclosed are methods and apparatus for performing automated lung slide detection using a computer device (e.g., an ultrasonic system). In some embodiments, these methods are performed by a computer device.

[0005] In some embodiments, a method for determining lung sliding includes generating attribute quality probabilities for a B-mode ultrasound image including the pleural line, and determining, based on the attribute quality probabilities, that the quality level of the B-mode ultrasound image is acceptable for determining the lung sliding. The method further includes generating one or more M-mode ultrasound images based on the B-mode ultrasound image, and generating one or more probabilities of lung sliding based on the one or more M-mode ultrasound images.

[0006] In some embodiments, a method for determining lung sliding includes generating a B-mode ultrasound image and generating an M-mode ultrasound image corresponding to the M-line. The method includes generating a probability of lung sliding at the M-line based on the M-mode ultrasound image and indicating the probability of lung sliding in at least one of the B-mode ultrasound images.

[0007] In some embodiments, a computer device implements an ultrasound system for determining lung sliding. In some embodiments, the computer device includes a memory that maintains B-mode ultrasound images and one or more M-mode ultrasound images, and a neural network at least partially implemented in the hardware of the computer device that generates one or more probabilities of lung sliding in one or more M-lines based on one or more M-mode ultrasound images. The computer device also includes a processor system that generates one or more M-mode ultrasound images corresponding to one or more M-lines based on pixels in the B-mode ultrasound images corresponding to one or more M-lines, and displays one or more representations of one or more probabilities of lung sliding in at least one of the B-mode ultrasound images.

[0008] The present invention will be better understood from the detailed description below and the accompanying drawings of various embodiments of the invention, but these drawings are for illustrative and illustrative purposes only and should not be construed as limiting the invention to any particular embodiment. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows several embodiments of an ultrasonic machine. [Figure 2A] This figure shows an example of a B-mode image. [Figure 2B] This figure shows an example of a B-mode image. [Figure 3A] This figure shows an example of a high-quality image. [Figure 3B] This figure shows an example of a low-quality image. [Figure 4] This figure shows an example of a pleural gland. [Figure 5A] This figure shows an M-mode image constructed from a series of M-line frames in B-mode video. [Figure 5B] This figure shows the processing of M-mode images using a neural network to generate the probability of lung sliding in three M-lines. [Figure 6] This figure shows several embodiments of a system that performs lung sliding detection processing. [Figure 7] This is a data flow diagram of several embodiments of the lung sliding detection process. [Figure 8] This is a flowchart of several embodiments of the process for generating M-mode ultrasound images from B-mode ultrasound images. [Figure 9A] This is a flowchart of several embodiments of the lung sliding determination process. [Figure 9B] This figure shows several embodiments of a lung sliding determination process that generates further probabilities of lung sliding and combines them with other probabilities of lung sliding. [Figure 10]This is a flowchart of several embodiments of a different lung sliding determination process. [Modes for carrying out the invention]

[0010] The following description provides numerous details to fully illustrate the present invention. However, it will be apparent to those skilled in the art that the present invention can be carried out without these specific details. In other cases, well-known structures and apparatus are shown in block diagram form rather than in detail, so as not to obscure the present invention.

[0011] This specification discloses a technique for automatically detecting lung sliding in ultrasound images generated using an ultrasound system. Lung sliding detection can be used to assist in the diagnosis of pneumothorax (PTX). Automated lung sliding detection based on ultrasound can improve diagnostic accuracy and speed, thereby reducing patient management time.

[0012] In some embodiments, the ultrasound system automatically detects lung sliding or non-sliding in ultrasound images through the use of one or more neural networks. These neural networks use models trained to determine lung sliding to reduce operator variability and implement a consistent lung sliding detection algorithm. In some embodiments, the neural network assists the user by acquiring acceptable quality video clips to determine the presence of sliding in the lungs.

[0013] The ability to diagnose PTX in real time by automatically detecting lung sliding using a portable ultrasound device can be life-saving because it allows for point-of-care diagnosis of PTX without the need to send the patient or images to a radiology department. Furthermore, automated lung sliding detection can improve diagnostic accuracy and speed, reducing patient management time.

[0014] Hereinafter, the automatic detection algorithm and its implementation example will be described in more detail.

[0015] FIG. 1 shows some embodiments of an ultrasonic machine including embodiments of the disclosed technology. Referring to FIG. 1, an ultrasonic transducer probe 100 includes an enclosure 110 extending between a distal end portion 112 and a proximal end portion 114. The ultrasonic transducer probe 100 is electrically coupled to an ultrasonic imaging system 130 via a cable 118 attached to the proximal end of the probe by a tension relief element 119. In some embodiments, the ultrasonic transducer probe 100 is electrically coupled wirelessly to the ultrasonic imaging system 130.

[0016] System electronics within the ultrasonic imaging system 130 are electrically coupled to a transducer assembly 120 having one or more transducer elements. The transducer assembly 120 transmits ultrasonic energy towards a subject from one or more transducer elements during operation and receives ultrasonic echoes from the subject. The ultrasonic echoes are converted into electrical signals by one or more transducer elements and transmitted electrically to the system electronics within the ultrasonic imaging system 130 to form one or more ultrasonic images.

[0017] Generally, capturing ultrasonic data from a subject using an exemplary transducer assembly (e.g., transducer assembly 120) includes generating ultrasonic waves, transmitting the ultrasonic waves to the subject, and receiving the ultrasonic waves reflected by the subject. Ultrasonic waves of a wide range of frequencies can be used for capturing ultrasonic data, for example, low-frequency ultrasonic waves (e.g., less than 15 MHz) and / or high-frequency ultrasonic waves (e.g., 15 MHz or more) can be used. A person skilled in the art can easily determine which frequency range should be used based on factors such as, for example, but not limited to, the depth of imaging and / or the desired resolution.

[0018] In some embodiments, to support the functions of the ultrasonic imaging system 130 in a manner well-known in the art, the ultrasonic imaging system 130 includes ultrasonic system electronics 134 that includes one or more processors, integrated circuits, ASICs, FPGAs, and power supplies. In some embodiments, the ultrasonic imaging system 130 also includes an ultrasonic control subsystem 131 having one or more processors. At least one processor, FPGA, or ASIC causes an electrical signal to be transmitted to the transducer(s) of the probe 100 to emit sound waves and receives an electrical pulse generated from the returning echo from the probe. One or more processors, FPGAs, or ASICs process the raw data associated with the received electrical pulse, form an image, and transmit it to the ultrasonic image subsystem 132, and the ultrasonic image subsystem 132 displays this image on the display screen 133. Accordingly, the display screen 133 displays an ultrasonic image from ultrasonic data processed by the processor of the ultrasonic control subsystem 131.

[0019] In some embodiments, the ultrasonic system can also have one or more user input devices (e.g., keyboard, cursor control device, microphone, camera, etc.) for inputting data and enabling the acquisition of measurements from the display of the ultrasonic display subsystem, a disk storage device (e.g., hard disk, floppy disk, thumb drive, compact disk (CD), digital video disk (DVD)) for storing the acquired images, and a printer for printing an image from the displayed data. These devices are not shown in FIG. 1 so as not to obscure the technology disclosed herein.

[0020] In some embodiments, the ultrasound system electronics 134 perform automated lung sliding detection. Automated detection of the presence or absence of lung sliding can assist clinicians in diagnosing or ruling out pneumothorax and includes benefits such as improved diagnostic accuracy and speed, reduced patient management time, and reduced operator variability, all of which are brought about by the use of a consistent lung sliding algorithm.

[0021] In some embodiments, automated lung sliding detection is performed using an automated artificial intelligence (AI) algorithm that determines whether sliding is present and its location within the body based on observation of multiple frames. In some embodiments, automated detection is performed by sending a series of images to a neural network (e.g., a convolutional neural network (CNN), a Swin Transformer, etc.). The series of images can be ultrasound video clips and can be sent as either a stack of images into a single CNN, a series of images into an RNN (recurrent neural network), or a time-based AI model that can provide an index (e.g., probability) of whether the images indicate the presence of lung sliding. Given appropriate training data with images fully annotated to indicate where sliding is present in each image, the model can learn to detect sliding and its location within images. In some embodiments, the automated detection process examines a single data line as opposed to examining frames as a whole. The single data line can be an M-line from an M-mode image. These M-mode images can be generated in several ways. For example, an M-mode image can be acquired through M-mode acquisition, which acquires a single set of data lines at a constant rate (e.g., 100 lines per second) over a certain period of time (e.g., 1 second is equal to 100 data lines). Alternatively, an M-mode image can also be acquired by creating an M-mode image from a B-mode image.

[0022] In some embodiments, the automated detection process detects lung sliding from a single M-mode strip (hereinafter, "M-strip") by creating one or more M-mode images based on one or more M-lines. That is, the M-strip is a series of three B-mode frames from which M-mode images are extracted at various M-lines. Details of these embodiments are described in more detail below. In some embodiments, the automated detection uses a neural network to examine a single M-strip to determine whether movement above and below the pleural line indicates that the lung is not collapsing. In some embodiments, if the acquisition frame rate is sufficiently high, the automated detection process extracts multiple M-strips from a group of B-mode images (e.g., a two-dimensional (2-D) video clip) and uses a neural network to detect lung sliding from the M-strips. In some embodiments, the automated detection process extracts M-mode lines from each B-mode image at a constant angle to the vertical using a technique often called anatomical M-mode, and uses a neural network to examine these lines to determine whether lung sliding is present. In either of these cases, the neural network has a model trained with appropriate training data, which includes images that are fully annotated to indicate where the sliding is located within each image, and learns to detect the sliding and its location in the input image.

[0023] Figures 2A and 2B show examples of a B-mode image with a selected horizontal position (shown in the upper center of the figure) and an M-strip of that position across multiple frames (shown below the B-mode image in the figure). In some embodiments, the M-strip is a three-dimensional data array (e.g., the x and y dimensions of the B-mode image, and the z dimension of time, i.e., frames). In some embodiments, the M-strip is extracted from a series of B-mode images, and the M-mode image is reconstructed from a two-dimensional ultrasound video clip.

[0024] An M-mode pattern that can indicate a lung exhibiting lung sliding (i.e., a lung showing a normal ventilation pattern during lung expansion and contraction) has a continuous horizontal line in the superficial layer of the pleural surface and contains a granular pattern deeper at this level. This is sometimes called the "seashore sign." Figure 2A shows a "seashore sign" in which sliding is detected in the pleural line 200 of the M-mode image 202 (generated from multiple frames of the B-mode image 201), with a transition 203 between "sea" and "beach" indicated by the upward and downward movement of the pleural line 200 in the B-mode image 201. In contrast, Figure 2B shows a pneumothorax (PTX) in the M-mode image 212 (generated from multiple frames of the B-mode image 211), which shows a pattern sometimes called the "stratosphere" or "barcode" sign 213, indicating no movement in the pleural line 210 in the B-mode image 210, and therefore no lung sliding.

[0025] Automatically detecting lung sliding by examining ultrasound images using neural networks has many advantages, including, but is not limited to, low computational requirements and ease of annotating the data (i.e., indicating whether or not sliding is occurring).

[0026] One challenge in automated detection processes using M-mode lines is determining which lines should be examined. In some embodiments, the determination of which lines to examine is made by identifying and examining a region of interest (ROI) within the image from which M-mode images should be extracted first (e.g., a region suitable for extracting M-lines). That is, the ROI represents a set of M-lines from which selection for M-mode image extraction is made. For example, if M-lines are selected at either an M-line location (i.e., an X-image location) between the left and right portions of the ROI, M-mode images are extracted from M-strips at these M-line (i.e., X) locations. In some embodiments, as described above, this ROI extends to the pleural line within the costal cavity of the lung. In one example, multiple M-lines from this region are examined to improve the accuracy of sliding determination. Also, different regions of the lung are thought to have different levels of sliding depending on the severity of the observed PTX.

[0027] Example of an automated detection embodiment In some embodiments, the automated detection process has several steps, including determining image quality for lung sliding detection, determining the ROI for lung sliding detection, determining acceptable image quality for the M-mode reconstruction region, and determining lung sliding detection. Each of these operations will be described in detail below.

[0028] Image quality and region of interest (ROI) determination for lung sliding detection To ensure that lung sliding detection is evaluated on acceptable images, an AI model, referred to herein as a neural network (e.g., CNN), is trained to recognize images with acceptable quality and appropriate views for use in automated lung sliding detection. In some embodiments, the determination of acceptable quality is based on one or more factors, including, but not limited to, resolution, gain, brightness, clarity, centrality, depth, pleural line recognition, and rib and / or rib shadow recognition.

[0029] In some embodiments, the neural network recognizes the appropriate view by recognizing images that have expected features such as pleural lines and ribs within the image. For example, in some embodiments, the neural network recognizes a distinct pleural line in the upper central region of the image and understands at least one rib shadow in one of the sides of the image. In one embodiment, the neural network is trained to recognize the location of the pleural line through different methods. These methods include, but are not limited to, the use of two points in the extension of the pleural line, left-right range and central depth, segmentation maps, and heat maps.

[0030] In some embodiments, data output from a neural network can be used in combination with heuristics to determine acceptable or good quality images, or unacceptable or low quality images. Figure 3A shows an example of a good quality image. The neural network can determine an image to be of low quality and unacceptable if, in addition to failing to recognize expected features such as pleural lines and ribs, it has one or more attributes such as being too dark, too bright, too blurry, too deep, too shallow, or off-center, and can determine an image to be of good quality and acceptable when it does not have these attributes that make the image unacceptable. Figure 3A shows an example of a good quality image. Figure 3B shows an example of a low quality image. In some embodiments, the neural network outputs good and bad quality indications as good / bad probabilities for multiple attributes.

[0031] In addition to calculating good / bad probabilities for multiple attributes, neural networks can also detect the positions (e.g., x, y positions or coordinates) of two points that represent the left and right edges or endpoints of a pleural line in an image. An example of a pleural line is shown in Figure 4. In Figure 4, markers 401 and 402 indicate the endpoints of the pleural line.

[0032] In some embodiments, good / bad probabilities generated by a neural network are used in combination with heuristic rules using the x / y positions of the pleural lines to determine the overall quality of a B-mode ultrasound image. In some embodiments, the x-position is used to determine whether the pleural lines extend to a specified distance in the image. In some embodiments, the specified distance is based on the proportion of the image centered on the central point of the image. For example, a line segment formed by connecting ROI points must cross the center of the image. If the pleural lines do not extend to the specified distance, the image is considered poor. The y-position of the pleural lines can be used to determine whether the image is too deep or too shallow. This positional information can be used to determine a region of interest (ROI) for calculating a metric for lung sliding. For example, the x-position of the pleural lines from the model can be used to determine an ROI that can be used to select the M-line position in the reconstructed M-mode image.

[0033] Determination of acceptable quality for the M-mode reconstruction region (M-strip) M-mode images can be reconstructed from M-strips. Before reconstructing M-mode images from M-strips, the frames are examined to determine whether the M-strips are acceptable for lung sliding detection. This determination may be based on the reported quality of each frame within the M-strip. In addition to or instead of this, in some embodiments, the lung sliding detection process examines ROI points to determine if the motion is too large. Excessive motion can make it difficult to determine the presence or absence of lung sliding in the reconstructed M-mode images. By searching for excessive motion, the M-strips are marked as good or poor quality. If the quality is poor, the M-strips are not used for lung sliding detection. In some embodiments, to detect motion within the M-strip frames, changes in the x,y position of the pleura in consecutive B-mode frames can be compared to each other to check whether they exceed a specified limit. If changes in the x,y position of the pleura exceed a specified limit, the motion in the M-strip frames is too large to be used for determining the presence or absence of lung sliding. Furthermore, a neural network can determine whether the overall motion in the B-mode image is too large to be used for lung sliding detection. For example, the neural network can examine the ROI on all frames, and if positional shifts are observed throughout the entire frame, it can determine that the M-mode image reconstructed from the B-mode image is not of sufficient quality.

[0034] Once an M-strip is designated as good quality, an M-mode image can be reconstructed for any of the M-lines in the B-mode image within the ROI. In some embodiments, an M-mode image can be reconstructed by selecting vertical image pixels of a given M-line sequence from each frame (e.g., 25 frames) within the M-strip. This process can be repeated for all selected frames. Combining these vertical sequences generates an M-mode image with a pulse repetition frequency (PRF) equal to the frame rate of the video clip.

[0035] Figure 5A shows an M-mode image constructed from M-line columns of B-mode video frames. In Figure 5A, B-mode video frames forming an M-strip 501 (e.g., 25 frames) are shown with the M-line columns 502 highlighted. An M-mode image 503 is created by combining the same columns of M-line columns 502 in each B-mode video frame 501. Although only three M-mode images 503 are shown in Figure 5A, fewer or more M-mode images 503 can be created from the M-line columns 502 of B-mode video frames 501. Lung sliding can be performed by evaluating multiple M-mode images constructed in this way. For example, a window with a pixel width of 3 or more pixels can be examined as a region of interest in the M-mode image 509. This window can be a sliding window that is examined to determine whether or not lung sliding exists somewhere in that region.

[0036] As described above, instead of or in addition to constructing M-mode images, lung sliding can also be detected by performing a lung sliding detection process on stored images (e.g., a CINE loop with a series of digital images from an ultrasound examination).

[0037] Judgment on lung sliding In some embodiments, a second neural network is trained to distinguish between M-mode images indicating lung sliding and M-mode images indicating the absence of lung sliding. Reconstructed M-mode images are fed into this model to determine the presence or absence of sliding. In some embodiments, this determination is based on only one M-mode image. In some embodiments, this determination is based on multiple M-mode images. For example, an ultrasound system can construct a variable number of M-mode images depending on available computational resources and response time, and pass these through a lung sliding model to determine the presence or absence of sliding. This detection can be performed for multiple M-mode images constructed from different M-line positions within an M-strip. This detection can also be performed for multiple M-strips (e.g., a different series of B-mode images that may or may not be temporally continuous). All lung sliding detection outputs can be combined to obtain a higher average accuracy than when examining lung sliding model detection from a single reconstructed M-mode. In some embodiments, an averaging function is used to combine the lung sliding detection outputs to achieve high accuracy.

[0038] Figure 5B illustrates the processing of an M-mode image to generate the probability of lung sliding in three M-lines using a neural network. Referring to Figure 5B, an M-mode image 510 is input to the neural network 510 to generate a B-mode image 511 in which an M-line 512 exists between pleural endpoints 521 and 522. The M-mode image 510 is an example of an M-mode image generated from an M-strip of a B-mode image, such as the M-mode image 503 in Figure 5A, and the M-line 512 is an example of an M-line selected from an M-strip, such as the M-line column 502 of the M-strip 501. While Figure 5B clearly shows that three M-mode images 510 are input to the neural network 510, in another embodiment, more or fewer M-mode images than three may also be input to the neural network to detect lung sliding.

[0039] In some embodiments, M lines 512 are displayed in the B-mode image 511 along with an indication of the probability of lung sliding. For example, one of the M lines 512 may be a specific color (e.g., green) indicating sliding, and another M line 511 may be displayed on the B-mode image 511 in a color (e.g., red) indicating a low or zero probability of lung sliding. In the example shown in Figure 5B, the number of M lines 512 displayed is equal to three. However, the technique described herein is not limited to displaying only three M lines. It is also possible that M lines 512 are present for all lines in the M-mode image 510. In such cases, these lines may indicate the start of lung sliding into a portion where lung sliding is not present. In addition to or instead of this, the user may select which M lines should be displayed in the B-mode image 511.

[0040] Example of a lung sliding detection system Figure 6 shows several embodiments of a system that performs lung sliding detection processing. Referring to Figure 6, a B-mode image 601 is provided to a quality check neural network (model) 602 and a region of interest neural network (model) 603. In one embodiment, the quality check neural network (model) 602 and the region of interest neural network (model) 603 are separate neural networks. In some embodiments, these neural networks are combined into a single neural network. In yet another embodiment, these networks share at least one common part and include other parts that are not shared between these networks.

[0041] The quality check neural network 602 receives the B-mode images 601 and determines whether each of the B-mode images 601 is of sufficient quality to be used in the lung sliding detection process. The quality check neural network 602 determines the quality as described above and outputs a quality level indicator 610 for each of the B-mode images. In some embodiments, the quality is output and displayed on a display screen (e.g., the display screen of an ultrasound machine) so that the user can guide and improve their image acquisition.

[0042] The region of interest neural network 603 receives the B-mode image 601 and determines the position 611 of the pleural line. The ROI neural network 603 outputs position information 611 for each of the B-mode images. In some embodiments, the position information includes a set of coordinates for the endpoints of the pleural line. In some embodiments, these coordinates are the x, y coordinates of the endpoints of the pleural line in each of the B-mode images.

[0043] Quality level indicator information 610 and location information 611 are input to the M-mode image generator 604 along with the B-mode image 601. The M-mode image generator 604 generates a reconstructed M-mode image 612 in response to these inputs. As described above, in some embodiments, the M-mode image generator 604 generates the reconstructed M-mode image 612 from the B-mode image. In addition to or instead of this, the M-mode image may also be acquired through a well-known M-mode image acquisition process.

[0044] The lung sliding detection neural network (model) 605 receives the reconstructed M-mode image 612 and performs lung sliding detection on the reconstructed M-mode image 612. In some embodiments, lung sliding detection is performed as described above. The lung sliding detection neural network 605 generates lung sliding detection results 613 as an output. In some embodiments, the lung sliding detection results 613 include the probability of lung sliding associated with each image. As described above, the lung sliding detection results can be displayed on an ultrasound image, such as a B-mode image. For example, an ultrasound system can display the lung sliding detection results as part of a heat bar as described above and / or as part of a binary icon that distinguishes the presence or absence of lung sliding, such as a thumbs up / thumbs down indicator.

[0045] One or more of the neural networks in Figure 6 can be implemented in many different ways. In one embodiment, the neural network includes a model that uses a sequence model including the EfficientNet architecture, a convolutional neural network (CNN), and / or a recurrent neural network (RNN). The detection techniques described herein can be implemented in conjunction with artificial intelligence (AI) or machine learning (e.g., adaptive boosting (adaboost), deep learning, supervised learning models, support vector machines (SVMs), gated recurrent units (GRUs), convolutional GRUs (ConvGRUs), long short-term memory (LSTMs), and line detection), and / or other preferred detection methods.

[0046] Example of a flow chart for the lung detection process Figure 7 is a data flow diagram of several embodiments of the lung sliding detection process. The process can be executed by processing logic that may include hardware (e.g., circuits, dedicated logic, memory, etc.), software (such as running on a general-purpose computer system or dedicated machine), firmware (e.g., software programmed into read-only memory), or a combination thereof. In some embodiments, the process is executed by one or more processors of a computer device, such as an ultrasound machine, which may include an ultrasound imaging subsystem, for example, but is not limited to these.

[0047] Referring to Figure 7, the process begins with processing logic (e.g., one or more memories) generating a B-mode ultrasound image (processing block 701). The processing logic generates one or more M-mode ultrasound images corresponding to one or more M-lines (processing block 702). In some embodiments, one or more M-mode images are generated based on the B-mode image and the pixels of one or more M-lines.

[0048] The processing logic generates one or more probabilities of lung sliding in one or more M-lines based on one or more M-mode ultrasound images (processing block 703). In one embodiment, the processing logic uses a neural network to generate one or more probabilities of lung sliding in one or more M-lines. In some embodiments, the neural network is implemented at least partially within the hardware of a computer device.

[0049] The processing logic generates one or more probabilities of lung sliding in one or more M lines, and then displays or otherwise indicates the representation of the lung sliding probabilities in at least one B-mode ultrasound image (processing block 704).

[0050] Figure 8 is a flowchart of several embodiments of a process for generating an M-mode ultrasound image from a B-mode ultrasound image. The process can be executed by processing logic that may include hardware (e.g., circuits, dedicated logic, memory, etc.), software (such as running on a general-purpose computer system or dedicated machine), firmware (e.g., software programmed into read-only memory), or a combination thereof. In some embodiments, the process is executed by one or more processors of a computer device such as an ultrasound machine, which includes, for example, an ultrasound imaging subsystem, but is not limited to these.

[0051] Referring to Figure 8, the process begins with the processing logic generating a quality level for the B-mode ultrasound image (processing block 801) and determining whether the quality level of the B-mode ultrasound image is higher than a quality threshold (processing block 802). In one embodiment, the processing logic generates a quality level for the B-mode ultrasound image based on attribute quality probabilities and coordinate pairs. Examples of attribute quality probabilities include resolution, gain, brightness, clarity, centrality, depth, pleural line recognition, and rib recognition. In some embodiments, the processing logic generates attribute quality probabilities for the B-mode image and coordinate pairs indicating the edges (e.g., endpoints) of the pleural line in the B-mode image. In some embodiments, the processing logic uses a neural network to generate attribute quality probabilities. In some embodiments, the neural network is implemented at least partially within the hardware of a computer device.

[0052] The processing logic determines whether the quality level is higher than the quality threshold, and then generates one or more M-mode ultrasound images (processing block 803). In some embodiments, the processing logic generates one or more M-mode ultrasound images in response to the quality level being higher than the quality threshold. In other words, M-mode ultrasound images are generated only if the quality of the B-mode image is higher than the quality threshold.

[0053] Subsequently, the processing logic generates one or more probabilities of lung sliding in one or more M lines of the M-mode ultrasound image (processing block 804). In some embodiments, the one or more probabilities are based on the M-mode ultrasound image generated from the B-mode ultrasound image.

[0054] Figure 9A is a flowchart of several embodiments of the lung sliding determination process. The process can be executed by processing logic that may include hardware (e.g., circuits, dedicated logic, memory, etc.), software (such as running on a general-purpose computer system or dedicated machine), firmware (e.g., software programmed into read-only memory), or a combination thereof. In some embodiments, the process is executed by one or more processors of a computer device, such as an ultrasound machine, which may include an ultrasound imaging subsystem, for example, but is not limited to these.

[0055] Referring to Figure 9A, the process begins by generating attribute quality probabilities for B-mode ultrasound images and coordinate pairs indicating the edges of pleural lines in the B-mode ultrasound images (processing block 901). In some embodiments, these attribute quality probabilities for B-mode ultrasound images and coordinate pairs are generated using a first neural network implemented at least partially within the hardware of a computer device.

[0056] Next, the processing logic determines the region of interest in the B-mode ultrasound image (processing block 902). In some embodiments, the region of interest in the B-mode ultrasound image is determined based on a previously generated coordinate pair.

[0057] Furthermore, the processing logic determines that the quality level of the B-mode ultrasound image is acceptable for lung sliding detection (processing block 903). In some embodiments, the determination that the B-mode ultrasound image has an acceptable quality level for lung sliding detection is based on previously generated attribute quality probabilities and the amount of motion in the region of interest. In some embodiments, the attribute quality probabilities represent the probability of at least one attribute quality selected from the group consisting of resolution, gain, brightness, clarity, centrality, depth, pleural line recognition, and rib recognition.

[0058] In some embodiments, determining that the quality level is acceptable involves determining the horizontal span of the pleura for each B-mode ultrasound image and comparing the horizontal span to a threshold distance. In some embodiments, determining the horizontal span of the pleura is performed based on the horizontal component of the coordinate pair. In some embodiments, the process includes processing logic that sets the threshold distance to a percentage of the size of at least one B-mode ultrasound image. For example, in some embodiments, to be considered good quality, the pleura must be vertically positioned over 20% to 60% of the image and the pleura must cross the center of the image. In some embodiments, determining that the quality level is acceptable involves determining the depth of each B-mode ultrasound image based on the vertical component of the coordinate pair.

[0059] The processing logic uses the B-mode ultrasound image to generate one or more M-mode ultrasound images corresponding to one or more M-lines in the region of interest (processing block 904). In some embodiments, the M-mode ultrasound image is derived from a series of pixels in each B-mode ultrasound image corresponding to one or more M-lines.

[0060] The processing logic generates the probability of lung sliding in one or more M-lines based on one or more M-mode ultrasound images (processing block 905). In some embodiments, the processing logic uses a neural network to generate the probability of lung sliding in one or more M-lines. The neural network can be implemented at least partially within the hardware of a computer device (e.g., an ultrasound machine such as the ultrasound system 130 in Figure 1).

[0061] The processing logic may also display a visual representation of one or more M lines indicating the probability of lung sliding in one or more M lines (processing block 906). The color-coded version of M line 512 shown in Figure 5B is an example of a visual representation of one or more M lines indicating probability by color. In some embodiments, the processing logic displays these M line representations within a B-mode ultrasound image. In some embodiments, the processing logic displays the visual representations horizontally across the region of interest via a heat bar or the like, as described above. In some embodiments, the process for generating the visual representations includes processing logic for filtering the probabilities. The probabilities can be filtered horizontally using a smoothing function.

[0062] In some embodiments, one or more M-mode ultrasound images comprise multiple M-mode ultrasound images, and one or more M-lines comprise multiple M-lines spanning the region of interest. In some embodiments, in such cases, the process generates a visual representation of the probability of lung sliding in the multiple M-lines and displays the visual representation horizontally across the region of interest.

[0063] In such cases, the processing logic can generate multiple M-mode ultrasound images based on a first start frame of the B-mode ultrasound image. In some embodiments, the process includes generating further M-mode ultrasound images based on a second start frame of the B-mode ultrasound image and generating further probabilities of lung sliding in multiple M lines. The process also includes combining the probabilities and further probabilities to form combined probabilities of lung sliding in multiple M lines. After forming the combined probabilities, the process generates and displays a visual representation of the combined probabilities. In some embodiments, the processing logic uses a neural network to generate further probabilities of lung sliding in multiple M lines based on the further M-mode ultrasound images.

[0064] Figure 9B shows several embodiments of a lung sliding determination process that generates further probabilities of lung sliding and combines them with other probabilities of lung sliding. The process can be carried out by processing logic that may include hardware (e.g., circuits, dedicated logic, memory, etc.), software (such as running on a general-purpose computer system or dedicated machine), firmware (e.g., software programmed into read-only memory), or a combination thereof. In some embodiments, the process is carried out by one or more processors of a computer device such as an ultrasound machine, which may include an ultrasound imaging subsystem, for example, but is not limited to these.

[0065] Referring to Figure 9B, the process begins by generating attribute quality probabilities for B-mode ultrasound images and coordinate pairs indicating the edges of pleural lines in the B-mode ultrasound images (processing block 911). In some embodiments, these attribute quality probabilities for B-mode ultrasound images and coordinate pairs are generated using a first neural network implemented at least partially within the hardware of a computer device.

[0066] Next, the processing logic determines the region of interest in the B-mode ultrasound image (processing block 912). In some embodiments, the region of interest in the B-mode ultrasound image is determined based on previously generated coordinate pairs.

[0067] Furthermore, the processing logic determines that the quality level of the B-mode ultrasound image is acceptable for lung sliding detection (processing block 913). In some embodiments, the determination that the B-mode ultrasound image has an acceptable quality level for lung sliding detection is based on previously generated attribute quality probabilities and the amount of motion in the region of interest. In some embodiments, the attribute quality probabilities represent the probability of at least one attribute quality selected from the group consisting of resolution, gain, brightness, clarity, centrality, depth, pleural line recognition, and rib recognition.

[0068] In some embodiments, determining that the quality level is acceptable involves determining the horizontal span of the pleura for each B-mode ultrasound image and comparing the horizontal span to a threshold distance. In some embodiments, determining the horizontal span of the pleura is performed based on the horizontal component of a coordinate pair. In some embodiments, the process includes processing logic that sets the threshold distance to a percentage of the size of at least one B-mode ultrasound image. For example, in some embodiments, to be considered good quality, the pleura must be vertically positioned over 20% to 60% of the image and the pleura must cross the center of the image. In some embodiments, determining that the quality level is acceptable involves determining the depth of each B-mode ultrasound image based on the vertical component of a coordinate pair.

[0069] The processing logic uses the B-mode ultrasound image to generate one or more M-mode ultrasound images corresponding to one or more M-lines in the region of interest (processing block 914). In some embodiments, the M-mode ultrasound image is derived from a series of pixels in each B-mode ultrasound image corresponding to one or more M-lines.

[0070] The processing logic generates a probability of lung sliding based on one or more M-mode ultrasound images (for example, in one or more M-lines) (processing block 915). In some embodiments, the processing logic uses a neural network to generate the probability of lung sliding in one or more M-lines. The neural network can be implemented at least partially within the hardware of a computer device (for example, an ultrasound machine such as the ultrasound system 130 in Figure 1).

[0071] Next, the processing logic generates a further M-mode ultrasound image based on a second start frame of the B-mode ultrasound image (processing block 916), and generates a further probability of lung sliding based on the further M-mode ultrasound image (processing block 917). In some embodiments, these are generated in the same manner as described above in relation to processing blocks 914 and 915.

[0072] The processing logic combines multiple probabilities generated from processing block 915 with further probabilities to form a combined probability of lung sliding (processing block 916).

[0073] The processing logic can also generate a visual representation of the combination probabilities (processing block 919) and display the visual representation (processing block 920). The color-coded version of M-line 512 shown in Figure 5B is an example of a visual representation of one or more M-lines indicating probabilities by color. In some embodiments, the processing logic displays these M-line representations within a B-mode ultrasound image. In some embodiments, the processing logic displays the visual representation horizontally across the region of interest via a heat bar or the like, as described above. In some embodiments, the process of generating the visual representation includes processing logic for filtering the probabilities. The probabilities can be filtered horizontally using a smoothing function.

[0074] Figure 10 is a flowchart of several embodiments of another lung sliding determination process. The process can be executed by processing logic that may include hardware (e.g., circuits, dedicated logic, memory, etc.), software (such as running on a general-purpose computer system or dedicated machine), firmware (e.g., software programmed into read-only memory), or a combination thereof. In some embodiments, the process is executed by one or more processors of a computer device such as an ultrasound machine, which may include an ultrasound imaging subsystem, for example, but is not limited to these.

[0075] Referring to Figure 10, the process begins by generating a B-mode ultrasound image (processing block 1001). In some embodiments, the B-mode ultrasound image is generated by a method well known in the art.

[0076] In some embodiments, processing logic determines the quality level of the B-mode ultrasound image (processing block 1002). In some embodiments, the processing logic determines the quality level using a process that includes generating coordinate pairs indicating the edges of the pleural lines in the B-mode ultrasound image, determining a region of interest in the B-mode ultrasound image based on the coordinate pairs, and determining the amount of motion in the region of interest. In some embodiments, the coordinate pairs indicating the edges of the pleural lines in the B-mode ultrasound image are generated using a neural network. This neural network may be added to a neural network that generates the probability of lung sliding in the M-line. In some embodiments, the neural network that generates the coordinate pairs is implemented at least partially within the hardware of the ultrasound system.

[0077] In some embodiments, the processing logic determines the quality level using a process that includes generating coordinate pairs indicating the edges of pleural lines in B-mode ultrasound images using a further neural network, at least partially implemented within the ultrasound system hardware. The quality level determination process may include determining the horizontal span of the pleural lines based on the coordinate pairs and comparing the horizontal span to a threshold distance. In some embodiments, the processing logic uses a neural network to generate coordinate pairs indicating the edges of pleural lines in B-mode ultrasound images. This neural network may be in addition to a neural network that generates the probability of lung sliding in the M-line. In some embodiments, the neural network that generates the coordinate pairs is at least partially implemented within the ultrasound system hardware.

[0078] In some embodiments, the processing logic determines the quality level using a process that includes generating attribute quality probabilities for B-mode ultrasound images, which represent the probability of at least one attribute quality selected from the group consisting of resolution, gain, brightness, clarity, centrality, depth, pleural line recognition, and rib recognition. In some embodiments, the processing logic generates attribute quality probabilities for B-mode ultrasound images using a neural network. This neural network may be in addition to a neural network that generates the probability of lung sliding in the M-line. In some embodiments, the neural network that generates attribute quality probabilities is implemented at least partially within the hardware of the ultrasound system.

[0079] Next, the processing logic discards a first portion of the B-mode ultrasound image based on its quality level (processing block 1003), while retaining a second portion of the B-mode ultrasound image based on its quality level (processing block 1004). In some embodiments, the probability of lung sliding is based on the retained portion of the B-mode ultrasound image. In some embodiments, the quality can also be displayed to the user.

[0080] The processing logic uses the retained B-mode ultrasound image to generate an M-mode ultrasound image corresponding to the M-line (processing block 1005). This process can be repeated to generate multiple M-mode ultrasound images. In some embodiments, the processing logic generates the M-mode image based on pixels in the B-mode ultrasound image corresponding to the M-line.

[0081] The processing logic generates the probability of lung sliding in each M-line based on the M-mode ultrasound image (processing block 1006). In some embodiments, the processing logic uses a neural network to generate the probability of lung sliding in the M-line. In some embodiments, the neural network is implemented at least partially within the hardware of the ultrasound system.

[0082] The processing logic generates the probability of lung sliding in the M-line, then generates further B-mode ultrasound images (processing block 1007), and shows the probability of lung sliding within the further B-mode ultrasound images (processing block 1008).

[0083] The systems, apparatus, and methods disclosed herein offer numerous advantages over conventional ultrasound systems, apparatus, and methods that assist in the diagnosis of PTX without implementing automated lung slide detection. For example, the ultrasound system disclosed herein can reliably diagnose PTX in real time using a portable ultrasound device, a diagnosis that cannot be easily performed with conventional ultrasound systems due to the time required to operate them and the errors caused by the operator. As a result, this ultrasound system can diagnose PTX more accurately and quickly than conventional ultrasound systems, and can have a life-saving effect in medical settings.

[0084] Furthermore, using the ultrasound system disclosed herein reduces the resource burden on care facilities compared to the use of conventional ultrasound systems. This is because the ultrasound system disclosed herein allows for the successful diagnosis of PTX using only the ultrasound system, without the need to refer patients to other imaging departments such as radiology. In contrast, conventional ultrasound systems, as described above, may not be able to adequately diagnose PTX and may require the use of further imaging, thus placing a higher burden on care facility resources than the ultrasound system disclosed herein. Therefore, the ultrasound system disclosed herein allows for more efficient operation of care facilities compared to conventional ultrasound systems, and thus enables the provision of better patient care.

[0085] Furthermore, the ultrasound system disclosed herein operates faster than conventional ultrasound systems that assist in the diagnosis of PTX without implementing automated lung slide detection, allowing operators to perform a wider range of ultrasound examinations within a given time compared to conventional ultrasound systems. Thus, by using the ultrasound system disclosed herein, patients can receive superior treatment compared to conventional ultrasound systems. Throughout this specification and the claims, words such as “comprise,” “comprising,” and “complement,” should be interpreted in a comprehensive sense, the opposite of exclusive or exhaustive, i.e., “including, but not limited to,” unless the context clearly requires another meaning. As used herein, the terms “connected,” “coupled,” and any variations thereof mean a direct or indirect connection or coupling between two or more elements, which may be physical, logical, or a combination thereof. Furthermore, when used in this application, the words “herein,” “above,” “below,” and similar words refer to the application as a whole and not to any particular part of it. Depending on the context, the singular or plural forms of words used in the above detailed description may each include plural or singular. The word “or” in relation to a list of two or more items covers all interpretations of the word: any of the items in the list, all of the items in the list, and any combination of the items in the list.

[0086] While the above description will likely reveal to those skilled in the art many variations and modifications of the present invention, it should be understood that any specific embodiment illustrated and described as an example is not intended to be considered limiting. Accordingly, references to the details of various embodiments are not intended to limit the claims, which describe only the features considered essential to the present invention. [Explanation of Symbols]

[0087] 100 Ultrasonic Transducer Probes 110 Enclosure 112 Distal end section 114 Proximal end portion 118 Cable 119 Tension relaxation element 120 Transducer Assembly 130 Ultrasonic System 131 Ultrasonic Control Subsystem 132 Ultrasound Imaging Subsystem 133 Display screen 134 Ultrasonic Systems Electronics

Claims

1. A method for determining lung sliding, performed by a computer device, To generate attribute quality probabilities for B-mode ultrasound images including pleural lines, Based on the attribute quality probability, it is determined that the quality level of the B-mode ultrasound image is acceptable for determining lung sliding. To generate one or more M-mode ultrasound images based on the aforementioned B-mode ultrasound image, The process involves generating one or more probabilities of lung sliding based on one or more M-mode ultrasound images, wherein the one or more M-mode ultrasound images include a plurality of M-mode ultrasound images, the one or more probabilities include a plurality of probabilities, and generating the plurality of M-mode ultrasound images is based on a first starting frame of the B-mode ultrasound image. A further M-mode ultrasound image is generated based on the second starting frame of the B-mode ultrasound image. To generate further probabilities of lung sliding based on the aforementioned further M-mode ultrasound images, The combination of the aforementioned multiple probabilities and the aforementioned further probabilities is used to form the combined probability of lung sliding, To generate a visual representation of the aforementioned combination probability, A method characterized by including the display of the aforementioned visual representation.

2. The method according to claim 1, wherein the attribute quality probability represents the probability of at least one attribute quality selected from the group consisting of resolution, gain, brightness, clarity, centrality, depth, pleural line recognition, and rib recognition.

3. Generating the attribute quality probability includes generating the attribute quality probability using a first neural network implemented at least partially within the hardware of the computer device, and generating the probability of one or more lung slidings includes generating the probability of one or more lung slidings using a second neural network implemented at least partially within the hardware of the computer device. The method according to claim 1.

4. To generate coordinates indicating the endpoints of the pleural line, Based on the aforementioned coordinates, the region of interest in the B-mode ultrasound image is determined, To determine the amount of movement in the aforementioned region of interest, Furthermore, determining that the quality level is acceptable is based on the amount of movement, The method according to claim 1.

5. To generate coordinates indicating the endpoints of the pleural line, Determining the horizontal span of the pleura based on the aforementioned coordinates, Comparing the aforementioned horizontal span with a threshold distance, Including the above, determining that the quality level is acceptable is based on the above comparison. The method according to claim 1.

6. The threshold distance is further set as a ratio of the size of at least one B-mode ultrasound image among the B-mode ultrasound images. The method according to claim 5.

7. The method further includes generating coordinates that indicate the endpoints of the pleural line, Determining that the quality level is acceptable includes determining the depth of each of the B-mode ultrasound images based on the coordinates. The method according to claim 1.

8. The method further includes extracting one or more pixel sequences corresponding to one or more M lines in each of the B-mode ultrasound images, The generation of the one or more M-mode ultrasound images is based on the one or more pixel sequences, and the one or more probabilities of the lung sliding correspond to the one or more M-lines. The method according to claim 1.

9. To generate one or more visual representations of the one or more M lines, each representing the probability of one or more lung sliding in the one or more M lines, Displaying one or more visual representations within at least one of the B-mode ultrasound images, The method according to claim 8, further comprising:

10. The plurality of M-mode ultrasound images correspond to a plurality of M-lines that cross the region of interest in the B-mode ultrasound image, the plurality of probabilities of lung sliding are those in the plurality of M-lines, and the method is To generate a visual representation of the multiple probabilities of lung sliding in the multiple M lines, Displaying the aforementioned visual representation horizontally across the area of ​​interest, The method according to claim 1, further comprising:

11. Generating the aforementioned visual representation includes filtering the plurality of probabilities horizontally using a smoothing function. The method according to claim 10.

12. A method for determining lung sliding, performed by a computer device, To generate B-mode ultrasound images, Determining the quality level of the B-mode ultrasound image, Discarding a first portion of the B-mode ultrasound image based on the quality level of the B-mode ultrasound image in the first portion, Maintaining the second portion of the B-mode ultrasound image based on the quality level of the B-mode ultrasound image in the second portion, Based on the pixels in the B-mode ultrasound image corresponding to the M-line, an M-mode ultrasound image corresponding to the M-line is generated. The process involves generating the probability of lung sliding in the M-line based on the M-mode ultrasound image, wherein generating the probability of lung sliding is based on the second portion of the B-mode ultrasound image. The probability of lung sliding is shown in at least one of the B-mode ultrasound images, A method characterized by including the following.

13. The generation of the M-mode ultrasound image is based on the pixels in the B-mode ultrasound image corresponding to the M-line. The method according to claim 12.

14. To generate a visual representation of the quality level of the B-mode ultrasound image, To guide the placement of the ultrasound probe, the visual representation is displayed within at least one of the B-mode ultrasound images. The method according to claim 12.

15. Determining the aforementioned quality level means To generate coordinates indicating the endpoints of the pleural lines in the B-mode ultrasound image, Based on the aforementioned coordinates, the region of interest in the B-mode ultrasound image is determined, To determine the amount of movement in the aforementioned region of interest, The method according to claim 12, including the method described in claim 12.

16. Determining the aforementioned quality level means To generate coordinates indicating the endpoints of the pleural lines in the B-mode ultrasound image, Determining the horizontal span of the pleura based on the aforementioned coordinates, Comparing the aforementioned horizontal span with a threshold distance, The method according to claim 12, including the method described in claim 12.

17. A computer device that implements an ultrasound system for determining lung sliding, A memory that maintains B-mode ultrasound images and one or more M-mode ultrasound images, A neural network, at least partially implemented within the hardware of the computer device, which generates one or more probabilities of lung sliding in one or more M lines based on one or more M-mode ultrasound images, Based on the pixels in the B-mode ultrasound image corresponding to the one or more M lines, one or more M-mode ultrasound images corresponding to the one or more M lines are generated. The processor system comprises one or more M-mode ultrasound images, one or more probabilities, and the generation of the multiple M-mode ultrasound images is based on a first start frame of the B-mode ultrasound images, and displays one or more representations of the one or more probabilities of lung sliding within at least one of the B-mode ultrasound images, and further, A further M-mode ultrasound image is generated based on the second starting frame of the B-mode ultrasound image. Based on the aforementioned further M-mode ultrasound images, a further probability of lung sliding is generated, The combination probability of lung sliding is formed by combining the aforementioned multiple probabilities and the further probabilities. A visual representation of the aforementioned combination probability is generated, A processor system that displays the aforementioned visual representation, A computer device characterized by being equipped with the following features.

18. The aforementioned processor system The quality level of the B-mode ultrasound image is generated, Determine whether the aforementioned quality level is higher than the quality threshold. In response to the quality level being higher than the quality threshold, one or more M-mode ultrasound images are generated. The computer device according to claim 17, as implemented in this manner.

Images