Automated Detection of Lung Sliding for Assisting the Diagnosis of Pneumothorax

Abstract

A method and apparatus for performing automated detection of a Lang slide using a computing device (e.g., an ultrasonic system, etc.) are disclosed. In some embodiments, the technique uses one or more neural networks to determine Lang sliding. In some embodiments, the neural network is part of a process for determining the probability of Lang sliding on one or more M-lines. In some embodiments, the technique displays the probability of one or more Lang slides in a B-mode ultrasound image.

Description

Technical Field

[0001] [Priority] This application claims the benefit of U.S. Non - Patent Application No. 18 / 140,526, filed on April 27, 2023, entitled "AUTOMATED DETECTION", and U.S. Provisional Patent Application No. 63 / 337,444, filed on May 2, 2022, entitled "AUTOMATED DETECTION OF LUNG SLIDE TO AID IN DIAGNOSIS OF PNEUMOTHORAX", and is incorporated herein by reference in its entirety.

[0002] The embodiments disclosed herein generally relate to ultrasonic imaging. More particularly, the embodiments disclosed herein relate to performing automated detection of lung sliding using an ultrasonic imaging system that includes generating a visualization (e.g., a three - dimensional image) indicative of the presence of lung sliding.

Background Art

[0003] Lung ultrasound (US) represents a new and promising proposal for diagnosing pneumothorax (PTX) with high sensitivity and specificity. More specifically, the determination of lung sliding or non - lung sliding can assist in the diagnosis of PTX, and the diagnosis of PTX using ultrasonic equipment has been carried out and is determined using lung sliding / non - lung sliding reference values. The reference values usually include the movement regarding the pleural line in ultrasonic images. Currently, clinicians evaluate B - mode video clips for the movement above and below the pleural line. In addition, clinicians use M - mode to view the movement above and below the pleural line. These techniques have the drawbacks that they must be performed by someone skilled in the art of recognizing lung sliding and / or are time - consuming and prone to user error. In certain situations, these drawbacks can prevent the use of these technologies in real time and can affect life-saving operations. SUMMARY OF THE INVENTION MEANS FOR SOLVING THE PROBLEM

[0004] A method and apparatus for implementing automated detection of lung sliding using a computing device (e.g., an ultrasound system, etc.) are disclosed. In some embodiments, the method is executed by a computing device. In some embodiments, the method receiving one or more B-mode ultrasound images including a pleural line, generating a feature list from the one or more B-mode ultrasound images, wherein the feature list indicates at least one feature of the pleural line, generating the feature list, and generating a probability of lung sliding using a neural network that is at least partially executed in the hardware of the computing device and is configured to process the B-mode ultrasound images of the feature list and the one or more B-mode ultrasound images, is executed by a computing device for lung sliding determination.

[0005] In some other embodiments, the method executed by a computing device for lung sliding determination is generating a B-mode ultrasound image, determining an instruction for improving the quality of the B-mode ultrasound image, and displaying the instruction on a user interface of the computing device. comprises. The method also Generating additional B-mode ultrasound images based on user adjustments executed based on the command, and Generating the likelihood of the Lang sliding using a neural network that is at least partially executed in the hardware of the computing device and is based on one or more of the additional B-mode ultrasound images, Comprising.

[0006] In some other embodiments, an ultrasound system for Lang sliding determination is A memory that maintains ultrasound images and a medical worksheet, A neural network that is at least partially executed in the hardware of the ultrasound system and generates the likelihood of the Lang sliding based on one or more of the ultrasound images, and A processing system that automatically and without user intervention presets columns of the medical worksheet using an indicator of the Lang sliding based on the likelihood generated by the neural network, Comprising.

[0007] Other systems, apparatuses, and methods for automated detection of Lang sliding are also described.

[0008] The present invention will be more fully understood from the detailed description given below and from the drawings of various embodiments of the invention attached hereto, but should not be received as an invention that limits the invention to specific embodiments, but is for explanation and understanding only.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2A

Figure 2B

Figure 3A

Figure 3B

Figure 4

Figure 5A

Figure 5B

Figure 6

Figure 7

Figure 8

Figure 9A

Figure 9B

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

DETAILED DESCRIPTION OF THE INVENTION

[0010] In the following description, numerous details are set forth in order to provide a more thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail in order to avoid obscuring the present invention.

[0011] The technology disclosed herein is capable of automatically detecting lung sliding in ultrasonic images generated by an ultrasonic system. The detection of lung sliding can be used to assist in the diagnosis of pneumothorax (PTX). Automated detection of lung sliding on US can improve diagnostic accuracy and speed, as well as reduce patient management time.

[0012] In some embodiments, the ultrasonic system automatically detects lung sliding or non-lung sliding in ultrasonic images through the use of one or more neural networks. These neural networks use models trained for lung sliding determination that help reduce operator-to-operator variability and perform a consistent algorithm for lung sliding detection. In some embodiments, the neural network assists the user by obtaining a video clip having a quality acceptable for determining the presence of sliding in the lungs.

[0013] The ability to diagnose PTX in real time with a portable ultrasound device through automatic lung sliding detection can have an impact on saving lives as the use of the ultrasound device would enable the diagnosis of PTX in the context of care without the need to send the patient or the image to the radiology department. Furthermore, automated lung sliding detection can improve the accuracy and speed of diagnosis, as well as reduce the patient management time.

[0014] In more detail below, automated detection algorithms and examples of implementation are discussed.

[0015] Figure 1 shows some embodiments of an ultrasound machine having the disclosed technology. Referring to Figure 1, the ultrasound transducer probe 100 includes a cover 110 that extends between a distal end 112 and a proximal end 114. The ultrasound transducer probe 100 is connected to an ultrasound imaging system 130 via a cable 118 that is attached to the proximal end of the probe by a strain relief element 119. In some embodiments, the ultrasound transducer probe 100 is electrically connected wirelessly to the ultrasound imaging system 130.

[0016] A transducer assembly 120 having one or more transducer elements is electrically connected to system electronics in the ultrasound imaging system 130. In operation, the transducer assembly 120 conducts ultrasonic energy from one or more transducer elements towards an object and receives ultrasonic echoes from the object. The ultrasonic echoes are converted into electrical signals by one or more transducer elements to form one or more ultrasonic images and are conducted electrically to the system electronics in the ultrasound imaging system 130.

[0017] Obtaining ultrasonic data from an object using an example of a transducer assembly (e.g., transducer assembly 120) generally includes generating ultrasonic waves, conducting the ultrasonic waves into the object, and receiving the ultrasonic waves reflected by the object. A wide range of ultrasonic frequencies can be used to obtain ultrasonic data. For example, low-frequency ultrasonic waves (e.g., 15 MHz) and / or high-frequency ultrasonic waves (e.g., greater than or equal to 15 MHz) can be used. A person skilled in the art can readily determine the frequency range for use based on factors such as, for example, the depth of imaging and / or the desired resolution, but is not limited thereto.

[0018] In some embodiments, the ultrasonic imaging system 130 includes ultrasonic system electronics 134 that includes one or more processors, integrated circuits, ASICs, FPGAs, and power supplies to support the functioning of the ultrasonic imaging system 130 in a manner well known in the art. In some embodiments, the ultrasonic imaging system 130 also includes an ultrasonic control subsystem 131 having one or more processors. At least one processor, FPGA, or ASIC causes an electrical signal to be sent to the transducer(s) probe 100 to emit acoustic waves and receives an electrical pulse from the probe created from the returning echo. One or more processors, FPGAs, or ASICs process the raw data associated with the received electrical pulse, form an image to be sent to the ultrasonic imaging subsystem 132, and display the image on the display screen 133. In this way, the display screen 133 displays an ultrasonic image from the ultrasonic data processed by the processor of the ultrasonic control subsystem 131.

[0019] In some embodiments, the ultrasonic system may also have one or more input devices (e.g., keyboard, cursor control device, microphone, camera, etc.) that allow data to be input and measurements to be obtained from the ultrasonic display subsystem, a disk storage device (e.g., hard, floppy, USB memory (tumb), compact disk (CD), digital video disk (DVDs)) for recording the acquired images, and a printer for printing images from the displayed data. These devices are not shown in FIG. 1 to avoid obscuring the technology disclosed herein.

[0020] In some embodiments, the ultrasonic system electronics 134 performs automated detection of lung sliding. Automated detection of whether lung sliding is present can assist clinicians in diagnosing or ruling out pneumothorax, with benefits such as improved diagnostic accuracy and speed, reduced patient management time, use of a non - conflicting algorithm for lung sliding, and reduced operator - to - operator variability in the results.

[0021] In some embodiments, the automated detection of lung sliding is performed by using an automated artificial intelligence (AI) algorithm for monitoring multiple frames to determine if sliding is present and where it is located within the body. In some embodiments, the automated detection is performed by sending a series of images to a neural network (e.g., convolutional neural network (CNN), Swin Transformer, etc.). The series of images can be an ultrasonic video clip, a collection of images stacked within a single CNN, a series of images in an RNN (Recurrent Neural Network), or also sent as a time-based AI model that can provide an indicator (e.g., probability) as to whether there is a lang sliding in the image. Appropriate training data including images fully annotated in each image as to where the sliding exists is provided, and the model can learn to detect the sliding and its location in the image. In some embodiments, as opposed to inspecting the entire frame, the automated detection process inspects a single line of data. The single line of data can be an M-line from an M-mode image. These M-mode images can be generated in a number of ways. For example, the M-mode image can be acquired through M-mode acquisition, and the collection of a single line of data is acquired at a fixed rate (e.g., 100 lines per second) for a certain period of time (e.g., equal to 100 data lines for 1 second). Additionally or alternatively, the M-mode image can be acquired by being created from a B-mode image.

[0022] In some embodiments, the automated detection process detects lang sliding from this single M-mode strip (hereinafter "M-strip") by creating one or more M-mode images based on one or more M-lines. That is, the M-strip is a sequence of B-mode frames (e.g., 3 B-mode frames) from which M-mode images are extracted with various M-lines. The details of these embodiments are described in more detail below. In some embodiments, to determine if there is movement at the upper and lower portions of the pleural line, indicating that the lung is not collapsed, the automated detection uses a neural network to examine a single M-strip. In some embodiments, if the frame rate acquisition is high enough, the automated detection process extracts multiple M-strips from a collection of B-mode images (e.g., a two-dimensional (2D) video clip, etc.) and uses a neural network to detect lung sliding from the M-strips. In some embodiments, the automated detection process, in a technique often referred to as anatomical M-mode, extracts M-mode lines at a vertical angle from each B-mode image and uses a neural network to examine these lines to determine if lung sliding is present. In both cases, the neural network has a trained model that uses appropriate training including images that are fully annotated with where the sliding exists in each image, and learns to detect the sliding and its location in the input image.

[0023] Figures 2A and 2B show an example of a B-mode image with a selected horizontal position (shown in the upper central part of the figure) and an M-strip at a position beyond the number of frames (shown below the B-mode image in the figure). In some embodiments, the M-strip is a three-dimensional array of data (e.g., dimensions x and y of the B-mode image in addition to the dimension z of time - i.e., frames). In some embodiments, the M-strip is extracted from a sequence of B-mode images, and the M-mode image is reconstructed from a 2-D ultrasound video clip.

[0024] A lung with lung sliding (i.e., a lung that exhibits a normal ventilation pattern in the inflated and deflated lungs) has a pattern on the deep granularity at this level and can be shown in the M-mode pattern of continuous horizontal lines on the shallow pleural surface. This is sometimes called the "sea shore sign". FIG. 2A shows a "sea shore sign" with a transition 203 between "sea" and "shore" where lung sliding is detected at the pleural line in an M-mode image 202 (generated from a number of frames of B-mode image 201) as indicated by the movement of the upper and lower parts of the pleural line 200 of B-mode image 201. In contrast, FIG. 2B shows a pneumothorax (PTX) having a pattern sometimes called the "stratosphere" or "bar code" sign 213 in an M-mode image 212 (generated from a number of frames from B-mode image 211), with no movement in B-mode image 211, thereby indicating no lung sliding at pleural line 210.

[0025] Using a neural network to automatically detect lung sliding by examining ultrasound images has a number of benefits including, but not limited to, its low computational requirements and ease of annotating data (e.g., sliding or not sliding).

[0026] One challenge with an automated detection process using an M-mode line is determining which line to test. In some embodiments, the determination of the line to test is done by first identifying a region of interest (ROI) in the image to be tested from which an M-mode image (e.g., an M-line suitable for extraction) is to be extracted and tested. For example, the ROI indicates a set of M-lines from the selection where the M-mode image is to be extracted. For example, any of the M-line positions (e.g., X image positions) between the left and right portions of the ROI, and once the M-line is selected, the M-mode images are extracted from that M-strip at those M-mode lines (e.g., X positions). In some embodiments, as discussed above, this ROI extends the pleural line in the rib space of the lung. In one example, one or more M-lines from the region are tested to improve the accuracy of the sliding determination. Different regions of the lung would also likely have different levels of sliding that depend on the severity of the PTX being monitored.

[0027] [Example of Automatic Detection Embodiment] In some embodiments, the automated detection process has many processes including determining the image quality for lung sliding detection, determining the ROI for lung sliding detection, determining the acceptable image quality for the M-mode reconstruction region, and determining the lung sliding detection. Each of these operations is described in more detail below.

[0028] <Determination of Image Quality and Region of Interest (ROI) for Lung Sliding Detection>[[]] To ensure that lung sliding detection is evaluated on acceptable images, an AI model, referred to herein as a neural network (e.g., CNN, etc.), can be trained to recognize images with acceptable quality and appropriate fields of view for use in automated detection of lung sliding. In some embodiments, the determination of acceptable quality is based on one or more factors including, but not limited to, resolution, gain, brightness, clarity, centrality, depth, recognition of the pleural line, and recognition of the ribs and / or rib shadows.

[0029] In some embodiments, the neural network recognizes an appropriate field of view by recognizing an image having features predicted as pleural lines and ribs in the image. For example, in some embodiments, the neural network recognizes a distinct pleural line in the upper central region of the image and sees at least one rib shadow on one side of the image. In one embodiment, the neural network is trained to recognize the position of the pleural line via different methods. These methods can include, but are not limited to, the use of two points on the extension of the pleural line, left - right extension and central depth, segmentation maps, and heat maps.

[0030] In some embodiments, the data output from the neural network can be combined with discovery techniques and used to determine whether an image is acceptable or of good quality, or to determine that an image is not acceptable or of low quality. Figure 3A shows an example of a good - quality image. In addition to, for example, not recognizing predicted features such as pleural lines and ribs, the neural network can determine that an image is not acceptable as having low quality for one or more of its characteristics such as being too dark, too bright, too blurry, too deep, too shallow, not centered; and the neural network can determine that an image is acceptable as having good quality if the image does not have any of these characteristics that make the image unacceptable. Figure 3A shows an example of a good - quality image. Figure 3A shows an example of a good - quality image. Figure 3B shows an example of a low - quality image. In some embodiments, the neural network outputs good / low probability indications of good quality and low quality for a number of characteristics.

[0031] In addition to calculating good / low probabilities for multiple features, the neural network can also detect the positions of two points (e.g., x, y positions or coordinates) that mark the start and right end, or terminus, of the pleural line in the image. Figure 4 shows an example of a pleural line. Referring to Figure 4, markers 401 and 402 indicate the termini of the pleural line.

[0032] In some embodiments, to determine the overall quality of a B-mode ultrasound image, the good / low probabilities generated by the neural network are used in combination with heuristic rules that utilize the x / y positions of the pleural line. In some embodiments, the x position is used to determine whether the pleural line spans a defined distance within the image. In some embodiments, the defined distance is based on the proportion of the image centered in the upper central portion of the image. For example, a line segment created by connecting ROI points can cross the center of the image. If the pleural line does not span the defined distance, then the image is considered to be of low quality. The y position of the pleural line can be used to determine if the image is too deep or too shallow. The position information can be used to make a determination that exceeds the region of interest (ROI) for which the metric for lung sliding is calculated. For example, the x position from the model to the pleural line can be used to determine an ROI that can be used to select an M-line position for a reconstructed M-mode image.

[0033] In some embodiments, the ultrasound system provides guidance or feedback to the user in terms of identifying features of the image that need to be adjusted to produce better image quality. For example, the ultrasound system can require that the image be placed more centrally and can indicate that the pleural line is too short or off the screen. Based on this guidance / feedback, the user can adjust the position of the probe to appropriately adjust the image. Examples of guidance also include adjusting depth up and down, adjusting gain, and moving left and right (e.g., to the center of the window, etc.). The feedback information 403 in FIG. 4 is an example of feedback or guidance that can be provided on the image or another part of the display. In this case, the feedback information 403 guides to position rather than adjust the position of the probe. The feedback / guidance information can be generated by a neural network. In some embodiments, without user intervention, if the neural network indicates that the field of view is good enough, the ultrasound system automatically activates, collects data, and analyzes the sliding (by providing the image to the model).

[0034] <Determination of acceptable quality for M-mode reconstruction region (M-strip)> The M-mode image can be reconstructed from the M-strip. Before constructing the M-mode image from the M-strip, the frame can be inspected to determine if the M-strip is acceptable for range sliding determination. This determination can be based on the quality of each reported frame in an M-strip of good quality. Additionally or alternatively, in some embodiments, the range sliding detection process inspects ROI points to determine if the movement is too large. Excessive movement can make it difficult to determine if there is range sliding in the reconstructed M-mode. By looking for excessive movement, M-strips are marked as having good or poor quality. If the quality of the M-strip is low, it will not be used for Lang sliding detection. In some embodiments, to detect movement within the M-strip frame, changes in the x, y positions of the pleural line in consecutive B-mode frames can be compared to each other to see if they exceed a defined limit. If the changes in the x, y positions of the pleural line exceed the defined limit, then the movement of the M-strip frame is sufficient for use in determining whether Lang sliding is present. It should be noted that in the B-mode image, this determination of whether it cannot be used for Lang sliding detection due to excessive overall movement can be made by a neural network. For example, a neural network can look at the ROI on multiple frames, and if there are adjustment defects of points everywhere in the frame, then the neural network will subsequently determine that the M-mode image reconstructed from the B-mode image is not of good enough quality.

[0035] Once an M-strip is selected as having good quality, then the M-mode image can be reconstructed for any of the M-lines in the B-mode image within the ROI. In some embodiments, the M-mode image can be reconstructed by obtaining the vertical pixels for the M-line columns given from each frame (e.g., 25 frames) in the M-strip. This process can be repeated for all of the selected frames. The combination of these vertical columns generates an M-mode image with a pulse repetition frequency (PRF) equal to the frame rate of the video clip.

[0036] Figure 5A shows an M-mode image composed of M-line columns of a B-mode video frame. Referring to FIG. 5A, a B-mode video frame (e.g., 25 frames, etc.) forming an M-strip 501 is shown with emphasized M-line columns 502. In each of the B-mode frames 501, the same columns of the M-line columns 502 can be combined to generate an M-mode image 503. While FIG. 5A shows only three M-mode images 503, it can be smaller or larger than the three M-mode images 503 created from the M-line columns 502 of the B-mode video frame 501. It should be noted that range sliding can be implemented by evaluating a plurality of M-mode images configured in this way. For example, a window with a width of 3 or more pixels can be inspected as a region of interest in the M-mode image 503. This window can be a sliding window that is inspected to make a determination as to whether range sliding exists at any position in that region.

[0037] As described above, instead of or in addition to constructing the M-mode image, the range sliding detection process can be executed, and range sliding can be detected on a recorded image (e.g., a CINE loop having a sequence of digital images from an ultrasonic examination, etc.).

[0038] To reduce the movement of the M-mode image, the ultrasonic system can remove the movement from the acquired B-mode frames used to construct the M-mode image. This movement removal can be implemented using algorithmic techniques such as, but not limited to, optical flow or motion detection algorithms, such as distortion or transformation. Once the movement is removed from the sequence of B-mode images, the M-mode image is then constructed.

[0039] <Judgment of lung sliding> In some embodiments, an additional (e.g., second) neural network is trained to distinguish between M-mode images that exhibit lung sliding and M-mode images that do not exhibit lung sliding. The reconstructed M-mode image can be provided to this model to determine whether there is sliding. In some embodiments, this determination is made based on only one M-mode image. In some embodiments, this determination is made based on a plurality of M-mode images. For example, depending on available computing resources and response time, the ultrasound system can construct a large number of M-mode images that vary, and pass them through a lung sliding model to determine whether there is sliding. This detection can be performed for a number of M-mode images composed of different M-line positions within an M-strip. This detection can also be performed for a number of M-strips (e.g., different sequences of B-mode images that can be consecutive or not within a time). All of the lung sliding detection outputs can be combined in this way to obtain a higher average accuracy than when looking at the lung sliding model detection from a single reconstructed M-mode. In some embodiments, the detection outputs of lung sliding are combined using an average value function to achieve high accuracy.

[0040] FIG. 5B shows processing of an M-mode image having a neural network for generating the probability of lung sliding with three M-lines. Referring to FIG. 5B, the M-mode image 510 is input into a neural network 520 for generating a B-mode image 511 having an M-line 512 between a pleural line end 521 and a pleural line end 522. The M-mode image 510 is an example of an M-mode image generated from an M-strip of a B-mode image, such as the M-mode image 503 in FIG. 5A, while the M-line 512 is an example of an M-line obtained from an M-strip, such as the M-line column 502 of the M-strip 501. FIG. 5B shows three M-mode images 510 input to the neural network 520. In an alternative embodiment, more or fewer than three M-mode images can be input to the neural network to detect lung sliding.

[0041] In some embodiments, the M-line 512 is displayed in the B-mode image 511 having an indicator indicating the presence or absence of the probability of lung sliding. For example, one of the M-lines 512 can be a gradient color (e.g., green) to indicate sliding, while another one of the M-lines 512 can be displayed on the B-mode image 511 having a gradient color (e.g., red) indicating that the probability of sliding is low or absent. In this case shown in FIG. 5B, the number of displayed M-lines 512 is equal to 3. However, the technique described herein is not limited to displaying only three M-lines. In the M-mode image 510, it should be noted that there is an M-line 512 for each line. In such a case, this line can indicate the start of lung sliding to a portion without lung sliding. Additionally or alternatively, the user can select which M-line is displayed in the B-mode image 511.

[0042] [Alternative Refinement of Lung Sliding Detection] <Automated - Refined Lung Sliding Detection in Doubtful Cases> If detected on the M-line where non-sliding is sparsely selected in the region of interest, the video clip can then be analyzed by further running the Lang sliding algorithm up to the number of M-lines (up to the number of M-lines in the ROI). In other words, the analysis can be enhanced by further analyzing the video clip by re-running the Lang sliding algorithm on a set of M-lines that are more dense rather than sparse. To filter out noise in the detection results, these Lang sliding detections can be averaged in the horizontal and / or temporal directions. This result can be displayed using a graph, such as a heat bar across the pleural line to show the probability of Lang sliding across the ROI. In some embodiments, one of the sets of M-mode images will be reconstructed starting from a given frame (known as the start frame) to create heat bar information. And each of the reconstructed M-mode images can be processed through a Lang sliding AI model to determine the probability of Lang sliding at its M-line (e.g., the M-line corresponding to the reconstructed M-mode image). These probabilities are then displayed as a heat bar, for example, where a red solid line is 100% non-sliding and a green solid line is 100% sliding. In some embodiments, all other probability values can be displayed as a gradient (gradation) between the red solid line and the green solid line. Additionally or alternatively, the ultrasound system can display the probability as a graph or an impulse response having a magnitude between zero and one. To smooth out noise in the probability, the probability can be filtered in the horizontal direction with a smoothing function.

[0043] To further enhance the likelihood faithfulness used to generate the heat bar, the above process can be repeated for two or more start frames, and then the likelihoods for each M-line from different start lines can be combined together to obtain a more faithful result. Combining algorithms can use a simple average, and the answer with a higher likelihood can be weighted more than the lower likelihood.

[0044] <Simultaneous images in B-mode and M-mode for higher faithful detection>

[0045] Additional imaging states can be created to enable a more highly faithful Lang's sliding determination. Instead of reconstructing a very low-resolution M-mode image from a B-mode frame as described above, the ultrasound system can create an imaging state with randomly placed additional M-mode pings between B-mode pings. These can be one selected M-line, and the pings can be conducted and received for the M-line. These pings can be obtained as fast as other B-mode pings and as slow as one additional M-mode ping per frame. The trade-off here would be the resolution of the M-mode image versus the frame rate of the B-mode video. In some embodiments, the determination for emitting any M-line is fixed, such as the center of the image or a ratio from the center of the image. Alternatively, in some embodiments, the determination for emitting any M-line, such as the center of the detected ROI or other positions within the ROI, is dynamically determined.

[0046] In some embodiments, multiple M-lines are selected to allow simultaneous acquisition of multiple high-resolution M-mode images, and randomly placed B-mode images are acquired. Assuming the acquisition of a single M-mode image scattered in the B-mode frame, there would be a trade-off between the number of M-lines and the frame rate of the B-mode image with respect to the temporary resolution of the M-mode image.

[0047] <High-resolution mapping model> In one example, the CNN can be trained to map a lower PRF rate image to a higher PRF rate by training a super-resolution neural network to construct a higher resolution image. The ultrasound system, as described above, can generate M-mode images of a first resolution from the M-strip and can run these M-mode images through a super-resolution neural network to create additional M-mode images having a resolution higher than the first resolution. These higher resolution M-mode images can be used as input for a Langs sliding detection model to generate a high-precision probability of Langs sliding. In some embodiments, the ultrasound system generates additional M-mode ultrasound images based on the M-mode ultrasound images, and the additional M-mode ultrasound images have a higher resolution than the M-mode ultrasound images. In some such embodiments, generating the probability of Langs sliding is based on the additional M-mode ultrasound images.

[0048] [Example of Langs sliding detection system] FIG. 6 shows some embodiments of a system for implementing a Langs sliding detection process. Referring to FIG. 6, the B-mode image from the B-mode image generator 601 is provided to a quality confirmation neural network (model) 602 and a region of interest neural network (model) 603. In one embodiment, the quality confirmation neural network (model) 602 and the region of interest neural network (model) 603 are separate neural networks. In some embodiments, these neural networks are combined into one neural network. In still other embodiments, these networks share at least one common part and include other parts that are not shared between these networks.

[0049] The quality confirmation neural network 602 receives the B-mode image from the B-mode image generator 601 and determines whether each B-mode image has sufficient quality in the longitudinal sliding detection process. The quality confirmation neural network 602 determines the quality as described above and outputs a quality level index 610 for each B-mode image. In some embodiments, the quality is output for display on a display screen (e.g., the display screen of an ultrasonic device, etc.) to guide the user and improve their image acquisition.

[0050] The region of interest neural network 603 receives the B-mode image from the B-mode image generator 601 and determines the position 611 of the pleural line. The ROI neural network 603 outputs position information 611 for each B-mode image. In some embodiments, the position information includes a set of coordinates at the end of the pleural line. In some embodiments, the coordinates are the x and y coordinates of the end of the pleural line in each B-mode image.

[0051] The quality level index information 610 and the position information 611 are input from the B-mode image generator 601 to the M-mode image generator 604 along with the B-mode image. In response to these inputs, the M-mode image generator 604 generates a reconstructed M-mode image 612. In some embodiments, the M-mode image generator 604 generates an M-mode image 612 reconstructed from the B-mode image, as described above. Additionally or alternatively, the M-mode image can be obtained through well-known M-mode image acquisition processes.

[0052] The Langs sliding detection neural network (model) 605 receives the reconstructed M-mode image 612 and performs Langs sliding detection on the reconstructed M-mode image 612. In some embodiments, the Langs sliding detection is performed as described above. As an output, the Langs sliding detection neural network 605 generates a Langs sliding detection result 613. In some embodiments, the Langs sliding detection result 613 includes probabilities for each image for Langs sliding. The Langs sliding detection result can be displayed on the ultrasonic image, for example, on the B-mode image described above, etc. For example, the ultrasonic system can display the Langs sliding detection result as part of a heat bar as described above and / or as part of a binary icon that distinguishes between Langs sliding and no Langs sliding, such as a thumbs up / thumbs down indicator.

[0053] One or more of the neural networks in FIG. 6 can be executed in a number of different ways. In one embodiment, the neural network includes an EfficientNet architecture, a model using a conventional convolutional neural network (CNN), and / or a sequence model having a recurrent neural network (RNN). The detection techniques described in this specification can be carried out in conjunction with artificial intelligence (AI) or machine learning (such as, for example, processing frame information in sequences and lines like adaptive boosting (adaboost), deep learning, supervised learning models, support vector machines (SVM), gated recurrent units (GRU), conventional GRU (ConvGRU), long short-term memory (LSTM), etc.), and / or another suitable detection method.

[0054] [Example flowchart of language detection processing] FIG. 7 shows a data flow diagram of some embodiments in the lang sliding detection process. It can be implemented by processing logic that can include hardware (such as circuitry, dedicated logic, memory, etc.), software (such as executed on a general-purpose computer system or a dedicated mechanical device, etc.), firmware (such as software programmed in read-only memory), or combinations thereof. In some embodiments, the processing can be carried out by one or more of the processors of a computing device, such as, for example, an ultrasonic mechanical device including an ultrasonic imaging subsystem, but is not limited thereto.

[0055] Referring to FIG. 7, the processing begins when processing logic (such as one or more memories) generates a B-mode ultrasonic image (processing block 701). The processing logic generates one or more M-mode ultrasonic images corresponding to one or more M-lines (processing block 702) In some embodiments, the one or more M-mode images are generated based on the pixels of the B-mode image and one or more M-lines.

[0056] The processing logic generates one or more probabilities of lang sliding on one or more M-lines based on the one or more M-mode ultrasonic images (processing block 703). In one embodiment, the processing logic uses a neural network to generate one or more probabilities of a Lang sliding on one or more M-lines. In some embodiments, the neural network is at least partially executed in the hardware of the computing device.

[0057] After generating one or more probabilities of a Lang sliding on one or more M-lines, the processing logic causes a representative of the probability of the Lang sliding to be displayed or otherwise presented in at least one B-mode ultrasound image (processing block 704).

[0058] FIG. 8 shows a flowchart of some embodiments in a process for generating an M-mode ultrasound image from a B-mode ultrasound image. It can be implemented by processing logic that can include hardware (e.g., circuitry, dedicated logic, memory, etc.), software (such as executed on a general-purpose computer system or a dedicated machine), firmware (e.g., software programmed in a read-only memory), or combinations thereof. In some embodiments, the processing is performed by one or more of the processors of a computing device, such as, but not limited to, an ultrasound machine including an ultrasound imaging subsystem.

[0059] Referring to FIG. 8, the process begins with the processing logic generating a quality level of the B-mode ultrasound image (processing block 801) and determining whether the quality level of the B-mode ultrasound image exceeds a quality threshold (processing block 802). In one embodiment, the processing logic generates a quality level of the B-mode ultrasound image based on a pair of probability quality characteristics and coordinates. Examples of probability quality characteristics include probabilities for quality characteristics including resolution, gain, brightness, clarity, centrality, depth, pleural line recognition, rib recognition, and other like characteristics. In some embodiments, the processing logic generates a pair of coordinates indicating the ends (e.g., terminations) of the pleural line in the B-mode image and the characteristics of the probability quality for the B-mode image. In some embodiments, the processing logic generates the characteristics of the probability quality using a neural network. In some embodiments, the neural network is executed on at least a portion of the hardware of the computing device.

[0060] After determining whether the quality level exceeds the quality threshold, the processing logic generates one or more M-mode ultrasound images (processing block 803). In some embodiments, the processing logic generates one or more M-mode ultrasound images according to a quality level exceeding the quality threshold. In other words, the M-mode ultrasound image is generated only when the quality of the B-mode image exceeds the quality threshold.

[0061] In some embodiments, the processing logic generates additional M-mode ultrasound images by conducting M-mode ultrasound signals interspersed in the B-mode ultrasound signals used to generate the B-mode ultrasound images.

[0062] Thereafter, the processing logic generates one or more probabilities of Langs sliding on one or more M-lines of the M-mode ultrasound image (processing block 804). In some embodiments, the one or more probabilities are based on the M-mode ultrasound images generated from the B-mode ultrasound images. In some embodiments, the one or more probabilities are also based on additional M-mode ultrasound images generated by conducting M-mode ultrasound signals interspersed in the B-mode ultrasound signals used to generate the B-mode ultrasound images. In one embodiment, the one or more probabilities based on the additional M-mode ultrasound are generated using a neural network.

[0063] Figure 9A shows a flowchart of some embodiments in the Lang sliding determination process. It can be implemented by processing logic that can include hardware (e.g., circuitry, dedicated logic, memory, etc.), software (such as executed on a general-purpose computer system or a dedicated mechanical device), firmware (e.g., software programmed in read-only memory), or combinations thereof. In some embodiments, the processing is implemented by one or more of the processors of a computing device, such as, but not limited to, an ultrasonic machine including, for example, an ultrasonic imaging subsystem.

[0064] Referring to Figure 9A, the processing begins by the processing logic generating (processing block 901) probabilistic quality characteristics for a B-mode ultrasonic image and a pair of coordinates indicating the ends of the pleural line in the B-mode ultrasonic image. In some embodiments, these probabilistic quality characteristics for the B-mode ultrasonic image and the pair of coordinates are generated by a first neural network that is at least partially executed in the hardware of the computing device.

[0065] Next, the processing logic determines (processing block 902) a region of interest in the B-mode ultrasonic image. In some embodiments, the region of interest in the B-mode ultrasonic image is determined based on the previously generated pair of coordinates.

[0066] The processing logic also determines (processing block 903) the quality level of the B-mode ultrasonic image as acceptable for Lang sliding determination. In some embodiments, the determination of a B-mode ultrasonic image having a quality level acceptable for Lang sliding determination is based on the previously generated probabilistic quality characteristics and the amount of movement in the region of interest. In some embodiments, the probability quality characteristic indicates the probability of at least one quality characteristic obtained from the group consisting of resolution, gain, brightness, clarity, centrality, depth, pleural line recognition, and rib recognition.

[0067] In some embodiments, for each B-mode ultrasound image, determining an acceptable quality level includes determining the total length of the horizontal pleural line and comparing the total length with a threshold distance. In some embodiments, determining the total length of the horizontal pleural line is performed based on the horizontal components of pairs of coordinates. In some embodiments, the process includes the processing logic setting a threshold distance that is a ratio of size in at least one B-mode ultrasound image. For example, in some embodiments, for good quality to be considered, the pleural line must be positioned between 20% and 60% of the image verticality and the pleural line should cross the middle of the image. In some embodiments, for each B-mode ultrasound image, determining an acceptable quality level includes determining the depth of each B-mode ultrasound image based on the vertical components of the coordinate pairs.

[0068] Using the B-mode ultrasound image includes the processing logic generating one or more M-mode ultrasound images corresponding to one or more M-lines in the region of interest (processing block 904). In some embodiments, the M-mode ultrasound image is from columns of pixels in each B-mode ultrasound image corresponding to one or more M-lines.

[0069] Based on the one or more M-mode ultrasound images, the processing logic generates a Langs sliding probability for the one or more M-lines (processing block 905). In some embodiments, the processing logic generates the Langs sliding probability for the one or more M-lines with a neural network. The neural network can be executed at least partially in the hardware of a computing device (e.g., an ultrasonic apparatus such as the ultrasonic system 130 in FIG. 1).

[0070] The processing logic can also display one or more visual representations (processing block 906) that indicate the probability of a Langsd sliding on one or more M-lines. The colored version of the M-line 512 depicted in FIG. 5B is an example of a visual representation in one or more M-lines that indicates the probability of having a color. In some embodiments, the processing logic displays the representation of these M-lines in the B-mode ultrasonic image. In some embodiments, the processing logic displays a visual representation that horizontally traverses the region of interest, such as via a heat bar, as previously described. In some embodiments, the process of generating a visual representation includes the processing logic filtering the probability. The probability can be filtered with a smoothing function in horizontal detection.

[0071] In some embodiments, the one or more M-mode ultrasonic images include a plurality of M-mode ultrasonic images, and the one or more M-lines include a plurality of M-lines that traverse the region of interest. In some embodiments, in such a case, the process generates a visual representation of the probability of a Langsd sliding with a plurality of M-lines and displays a visual representation that horizontally traverses the region of interest.

[0072] In such a case, the processing logic can generate a plurality of M-mode ultrasonic images based on a first start frame of the B-mode ultrasonic image. In some embodiments, the process includes generating additional M-mode ultrasonic images based on a second start frame of the B-mode ultrasonic image and generating additional probabilities of a Langsd sliding with a plurality of M-lines. The process can also include combining a probability with an additional probability to form a combined probability of rungs sliding on multiple M-lines. After forming the combined probability, the process generates and displays a visual representation of the combined probability. In some embodiments, the processing logic generates, with a neural network, an additional probability of rungs sliding on multiple M-lines based on an additional M-mode ultrasound image.

[0073] FIG. 9B shows some embodiments of a process for rung sliding determination where an additional probability of rung sliding is generated and combined with other probabilities of rung sliding. It can be implemented by processing logic that can include hardware (e.g., circuitry, dedicated logic, memory, etc.), software (such as executed on a general-purpose computer system or a dedicated machine device), firmware (e.g., software programmed in read-only memory), or combinations thereof. In some embodiments, the process is implemented by one or more processors of a computing device, such as, but not limited to, an ultrasound machine including an ultrasound imaging subsystem.

[0074] Referring to FIG. 9B, the process begins by the processing logic generating (processing block 911) a probability quality characteristic for a B-mode ultrasound image and a pair of coordinates indicating the ends of the pleural line in the B-mode ultrasound image. In some embodiments, the probability quality characteristic for the B-mode ultrasound image and the pair of coordinates is generated by a first neural network that is at least partially executed in the hardware of the computing device.

[0075] Next, the processing logic determines a region of interest in the B-mode ultrasound image (processing block 912). In some embodiments, the region of interest in the B-mode ultrasound image is determined based on a pair of previously generated coordinates.

[0076] The processing logic also determines the quality level of the B-mode ultrasound image as acceptable for line sliding determination (processing block 913). In some embodiments, the determination of a B-mode ultrasound image having a quality level acceptable for line sliding determination is based on a previously generated quality characteristic of probability and the amount of movement in the region of interest. In some embodiments, the quality characteristic of probability indicates the probability of at least one quality characteristic obtained from the group consisting of resolution, gain, brightness, clarity, centrality, depth, recognition of the pleural line, and recognition of the ribs.

[0077] In some embodiments, for each B-mode ultrasound image, determining the quality level as acceptable includes determining the total length of the horizontal line of the pleural line and comparing the total length with a threshold distance. In some embodiments, determining the total length of the horizontal line of the pleural line is performed based on the horizontal components of the pair of coordinates. In some embodiments, the process includes the processing logic setting a threshold distance that is a percentage of the size in at least one B-mode ultrasound image. For example, in some embodiments, for good quality to be considered, the pleural line should be located between 20% and 60% of the image verticality, and the pleural line should cross the middle of the image. In some embodiments, for each B-mode ultrasound image, determining the quality level as acceptable includes determining the depth of each of these B-mode ultrasound images based on the vertical components of the coordinate pairs.

[0078] Using the B-mode ultrasound image includes the processing logic generating one or more M-mode ultrasound images corresponding to one or more M-lines in the region of interest (processing block 914). In some embodiments, the M-mode ultrasound image is from a column of pixels in each B-mode ultrasound image corresponding to one or more M-lines.

[0079] Based on the one or more M-mode ultrasound images, the processing logic generates a likelihood of a Langs sliding based on the one or more M-mode images (e.g., one or more M-lines) (processing block 915). In some embodiments, the processing logic generates the likelihood of a Langs sliding with a neural network for one or more M-lines. The neural network can be at least partially executed in the hardware of a computing device (e.g., an ultrasound machine such as the ultrasound system 130 in FIG. 1).

[0080] The processing logic then generates an additional M-mode ultrasound image based on a second start frame of the B-mode ultrasound image (processing block 916) and an additional likelihood of a Langs sliding based on the additional M-mode ultrasound image (processing block 917). In some embodiments, in the integration of processing block 914 and processing block 915, it is generated in the same manner as described above.

[0081] The processing logic combines the plurality of likelihoods generated from processing block 915 having additional likelihoods to form a combined likelihood of a Langs sliding (processing block 918).

[0082] The processing logic can also generate a visual representation of the combined likelihood (processing block 919) and display the visual representation (processing block 920). The color-coded version of the M-line 512 depicted in FIG. 5B is an example of a visual representation in one or more M-lines indicating likelihoods with colors. In some embodiments, the processing logic displays representatives of these M-lines in a B-mode ultrasound image. In some embodiments, the processing logic displays a visual representation that horizontally traverses the region of interest, such as via a heat bar, as previously described. In some embodiments, the process of generating a visual representation includes the processing logic filtering probabilities. Probabilities can be filtered with a smoothing function in horizontal detection.

[0083] FIG. 10 shows a flowchart of some embodiments in a process for another Langs sliding determination. It can be implemented by processing logic that can include hardware (e.g., circuitry, dedicated logic, memory, etc.), software (such as executed on a general-purpose computer system or a dedicated machine), firmware (e.g., software programmed in read-only memory), or combinations thereof. In some embodiments, the process is implemented by one or more of the processors of a computing device, such as, but not limited to, an ultrasound machine including an ultrasound imaging system.

[0084] Referring to FIG. 10, the process begins by the processing logic generating a B-mode ultrasound image (processing block 1001). In some embodiments, the B-mode ultrasound image is generated in a manner well known in the art.

[0085] In some embodiments, the processing logic determines the quality level of the B-mode ultrasound image (processing block 1002). In some embodiments, the processing logic determines the quality level using a process that includes generating a pair of coordinates indicating the ends of the pleural line in the B-mode ultrasound image, determining a region of interest in the B-mode ultrasound image based on the pair of coordinates, and determining the amount of movement in the region of interest. In some embodiments, the pair of coordinates indicating the ends of the pleural line in the B-mode ultrasound image is generated by a neural network. The neural network can be added to a neural network that generates the probability of Lang sliding on the M-line. In some embodiments, the neural network that generates the pair of coordinates is at least partially executed in the hardware of the ultrasound system.

[0086] In some embodiments, the processing logic determines the quality level using a process that includes generating, by an additional neural network at least partially executed in the hardware of the ultrasound system, a pair of coordinates indicating the ends of the pleural line in the B-mode ultrasound image. The process for determining the quality level can also include determining the total horizontal length of the pleural line based on the pair of coordinates and comparing the total horizontal length with a threshold distance. In some embodiments, the processing logic generates a pair of coordinates indicating the ends of the pleural line in the B-mode ultrasound image using a neural network. The neural network can be added to a neural network that generates the probability of Lang sliding on the M-line. In some embodiments, the neural network that generates the pair of coordinates is at least partially executed in the hardware of the ultrasound system.

[0087] In some embodiments, the processing logic determines a quality level using a process that includes generating probabilistic quality characteristics for a B-mode ultrasound image indicative of the probability of at least one quality characteristic taken from a group consisting of resolution, gain, brightness, clarity, centricity, depth, pleural line recognition, and rib recognition. In some embodiments, the processing logic generates probabilistic quality characteristics for a B-mode ultrasound image using a neural network. The neural network can be augmented with a neural network that generates the probability of line sliding in the M-line. In some embodiments, the neural network that generates probabilistic quality characteristics is at least partially executed in the hardware of the ultrasound system.

[0088] Thereafter, the processing logic excludes a first portion of the B-mode ultrasound image based on the quality level of the B-mode ultrasound image (processing block 1003), while maintaining a second portion of the B-mode ultrasound image based on those quality levels (processing block 1004). In some embodiments, the probability of line sliding is based on the retained portion of the B-mode ultrasound image. In some embodiments, the quality should also be noted as being displayed to the user.

[0089] By using the retained B-mode ultrasound image, the processing logic generates an M-mode ultrasound image corresponding to the M-line (processing block 1005). It should be noted that this process can be repeated in such a manner that a plurality of M-mode ultrasound images are generated. In some embodiments, the processing logic generates an M-mode image based on pixels in a B-mode ultrasound image corresponding to the M-line.

[0090] In some embodiments, the processing logic also generates additional M-mode ultrasound images based on the M-mode ultrasound images. The additionally generated M-mode ultrasound images have a higher resolution than the constructed low-resolution M-mode ultrasound images, and the probability of Langs sliding is based on the additional M-mode ultrasound images. In some embodiments, generating the additional M-mode ultrasound images is performed using a neural network, such as a super-resolution neural network. A neural network (e.g., a super-resolution neural network) can be added to the neural network that generates the probability of Langs sliding on the M-line. In some embodiments, the neural network (e.g., a super-resolution neural network) is at least partially executed in the hardware of the ultrasound system.

[0091] Based on the M-mode ultrasound images, the processing logic generates the probability of Langs sliding on each M-line (processing logic 1006). In some embodiments, the processing logic uses a neural network to generate the probability of Langs sliding on the M-line. In some embodiments, the neural network is at least partially executed in the hardware of the ultrasound system.

[0092] After generating the probability of Langs sliding on each M-line, the processing logic generates an additional B-mode ultrasound image (processing block 1007) and indicates the probability of Langs sliding in the additional B-mode ultrasound image (processing block 1008).

[0093] In some embodiments, Langs sliding detection generates the probability of Langs sliding based on the B-mode. In some embodiments, the ultrasonic system uses a neural network that is at least partially executed in the hardware of the ultrasonic system and is based on a B-mode ultrasonic image. In some embodiments, the neural network generates a probability of lung sliding by operating on a clip of the B-mode image. For example, the neural network can be given features that deviate from a model (e.g., a QC (QCRY) model), and the features of the B-mode image are generated from the layers of the QC model. These features can be used as conditional / additional / secondary inputs with the B-mode image as the main input. In some embodiments, the system selects a B-mode image to reduce the load on the neural network. For example, the system can exclude redundant images that would make the execution of the neural network slower. In some embodiments, the neural network is given only the region of interest (ROI) of each B-mode image instead of the entire image. The ROI can be based on a provided width and / or height. The ROI can be selected to obtain the pleural line. When specifying the width and / or height of the ROI of interest, the specified position can indicate the distance from the end of the pleural line to the edge of the image in both the horizontal and vertical directions (left and / or right of the pleural line and above and / or below the pleural line). This results in the pleural line being centered in the ROI. Additionally or alternatively, the height and width of the ROI can be based on distances specified by the neural network or by the ultrasonic device.

[0094] FIG. 11 shows a flowchart in some other embodiments for the lung sliding determination process. It can be implemented by processing logic that can include hardware (e.g., circuitry, dedicated logic, memory, etc.), software (such as executed on a general-purpose computer system or dedicated machinery), firmware (e.g., software programmed in read-only memory), or combinations thereof. In some embodiments, the processing is performed by one or more of the processors of a computing device, such as, but not limited to, an ultrasonic apparatus including an ultrasonic imaging system.

[0095] Referring to FIG. 11, the processing begins with the processing logic receiving one or more B-mode ultrasonic images including a pleural line (processing block 1101), and creating a feature list indicating at least one of the features of the pleural line from the one or more B-mode ultrasonic images (processing block 1102).

[0096] After generating the feature list, the processing logic uses a neural network, at least partially executed in the hardware of the computing device and configured to process the feature list and the B-mode ultrasonic images of the B-mode ultrasonic images, to generate the probability of Lang sliding (processing block 1103). In some embodiments, generating the probability of Lang sliding includes activating a neural network for processing the feature list and the B-mode ultrasonic images automatically and without user intervention based on B-mode ultrasonic images having a quality level exceeding a threshold quality level.

[0097] In some embodiments, the processing revealed in FIG. 11 further includes determining a region of interest for the B-mode ultrasonic image based on the position of the pleural line in the B-mode ultrasonic image. In some such embodiments, generating the likelihood of lung sliding is not based on additional pixels of the B-mode ultrasound image not included in the region of interest, but rather on the pixels of the B-mode ultrasound image included in the region of interest. In some embodiments, the position indicates the distance from the end of the pleural line to the end of the B-mode ultrasound image, and determining the region of interest is based on the distance.

[0098] In some embodiments, the process revealed in FIG. 11 further includes generating an additional likelihood of lung sliding based on additional B-mode ultrasound images in one or more B-mode ultrasound images. In some such embodiments, generating the additional likelihood of lung sliding is based on a feature list.

[0099] In some embodiments, the process revealed in FIG. 11 further includes generating an additional likelihood of lung sliding based on additional B-mode ultrasound images in one or more B-mode ultrasound images, generating an additional feature list from the additional B-mode ultrasound images, and generating the additional likelihood of lung sliding is based on the additional feature list.

[0100] In some embodiments, the process revealed in FIG. 11 further includes generating an additional likelihood of lung sliding based on additional B-mode ultrasound images in one or more B-mode ultrasound images, merging the likelihood and the additional likelihood to form a combined likelihood of lung sliding, and displaying a representation of the combined likelihood of lung sliding on a user interface of a computing device.

[0101] In some embodiments, the process revealed on FIG. 11 further determines additional B-mode ultrasound images in one or more B-mode ultrasound images as redundant to the B-mode ultrasound image, and to prevent the neural network from processing the additional B-mode ultrasound images, includes excluding additional B-mode ultrasound images from one or more B-mode ultrasound images.

[0102] FIG. 12 shows a flowchart in some other embodiments in the Langs sliding determination process. It can be implemented by processing logic that can include hardware (e.g., circuitry, dedicated logic, memory, etc.), software (such as executed on a general-purpose computer system or a dedicated mechanical device), firmware (e.g., software programmed in read-only memory), or combinations thereof. In some embodiments, the process is implemented by one or more of the processors of a computing device, such as, but not limited to, an ultrasound mechanical device including an ultrasound imaging system.

[0103] Referring to FIG. 12, the process starts when the processing logic generates a B-mode ultrasound image (processing block 1201). In some embodiments, the B-mode ultrasound image includes a pleural line, and the quality of the B-mode ultrasound image is based on the position of the pleural line in the B-mode ultrasound image.

[0104] After generating the B-mode ultrasound image, the processing logic determines (processing block 1202) instructions for improving the quality of the B-mode ultrasound image and displays the instructions on the user interface of the computing device (processing block 1203). In some embodiments, the instructions include at least one guidance for moving an ultrasonic probe, adjusting imaging parameters, and selecting a neural network from a list of neural networks available on a computing device.

[0105] Based on user adjustments performed based on the instructions, the processing logic generates one or more additional B-mode ultrasonic images (processing block 1204). In some embodiments, when generating a plurality of B-mode ultrasonic images, the process also includes determining redundant B-mode ultrasonic images in the plurality of B-mode ultrasonic images and excluding one or more redundant B-mode ultrasonic images by preventing the neural network from processing data determined from the one or more redundant B-mode ultrasonic images, thereby generating a likelihood of Langs sliding.

[0106] After generating one or more additional B-mode ultrasonic images based on user adjustments performed based on the instructions, the processing logic generates a likelihood of Langs sliding using a neural network based on the one or more additional B-mode ultrasonic images (processing block 1205). In some embodiments, the neural network is at least partially executed in the hardware of the computing device. In some embodiments, generating a likelihood of Langs sliding includes automatically activating the neural network based on one or more additional B-mode ultrasonic images having a quality level exceeding a threshold quality level and without user intervention.

[0107] In some embodiments, the process revealed in FIG. 12 further includes determining regions of interest in one or more additional B-mode ultrasonic images. In some such embodiments, generating the likelihood of lung sliding is not based on additional pixels of one or more additional B-mode ultrasound images that are not included in the region of interest, but rather is based on pixels of one or more additional B-mode ultrasound images that are included in the region of interest. In some such embodiments, determining the region of interest is based on the pleural line in one or more additional B-mode ultrasound images.

[0108] In some embodiments, a computing system (e.g., an ultrasound system) that performs the lung sliding detection described herein includes an enhanced workflow and user interface. In this case, the computing system pre-populates a worksheet using data from a neural network or AI-based mechanism. The worksheet is then displayed to or otherwise provided to the clinician for verification (e.g., sending a message such as "Verify based on self-assessment"). In one embodiment, the ultrasound system includes an ultrasound probe having an inertial measurement unit (IMU), and the protocol detects where the scan is and then performs the auto-population described above. The clinician typically follows a specific pattern according to the protocol, and based on the IMU data, the ultrasound system can determine that they are moving from L1 to L2. In some embodiments, the user interface can indicate the detection of a lung pulse.

[0109] FIG. 13 shows a flowchart in some other embodiments for a lung sliding determination process using the enhanced workflow described above. It can be implemented by processing logic that can include hardware (e.g., circuitry, dedicated logic, memory, etc.), software (such as executed on a general-purpose computer system or a dedicated mechanical device), firmware (e.g., software programmed in read-only memory), or combinations thereof. In some embodiments, the processing is performed by one or more of the processors of a computing device, such as, but not limited to, an ultrasonic mechanical device including, for example, an ultrasonic imaging system.

[0110] Referring to FIG. 13, the processing begins with the processing logic maintaining an ultrasonic image and a medical worksheet in the memory of the ultrasonic system (processing block 1301) and generating a probability of Langs sliding based on one or more ultrasonic images (processing block 1302). In some embodiments, the processing logic uses a neural network that is at least partially executed in the hardware of the ultrasonic system to generate the probability of Langs sliding.

[0111] After generating the probability of Langs sliding, the processing logic pre-sets, automatically and without user intervention, a column of the medical worksheet having an index of Langs sliding based on the probability (processing block 1303). In some embodiments, the processing logic inputs a column of the medical worksheet having an index of Langs sliding according to the neural network that generates the probability.

[0112] In some embodiments, the processing revealed in FIG. 13 further includes displaying, within a display screen (e.g., an ultrasonic system display screen), a medical worksheet that includes a pre-set column having an index of Langs sliding and a request for user confirmation in the index of Langs sliding (processing block 1304).

[0113] In some embodiments, the process revealed on FIG. 13 further includes generating position data, and determining the columns of the medical worksheet is based on the position data. In some such embodiments, the position data is generated by a sensor in the ultrasonic probe of the ultrasonic system.

[0114] In some embodiments, the process revealed on FIG. 13 further includes scanning a lung region using the ultrasonic probe of the ultrasonic system based on the position data generated by the position sensor of the ultrasonic probe. These lung regions can be shown in the medical worksheet. In some embodiments, the ultrasonic system also includes an ultrasonic probe having a position sensor configured to generate position data. In such a case, the processing system can be executed to indicate in the medical worksheet the lung region scanned using the ultrasonic probe based on the position data.

[0115] FIG. 14 shows an example of a user interface that can be displayed for a person (e.g., a clinician) to use and / or view on a display on an ultrasonic device.

[0116] The systems, devices, and methods disclosed herein constitute a number of advantages over conventional ultrasonic systems, devices, and methods that do not perform automated detection of lung sliding to assist in the diagnosis of PTX. For example, the ultrasonic system disclosed herein can reliably diagnose PTX in real time using a portable ultrasonic device, which cannot simply be done using a conventional ultrasonic system due to the time required to operate the conventional ultrasonic system and the errors induced by the operator. As a result, the ultrasonic system can diagnose PTX more accurately and faster than a conventional ultrasonic system, and can have an impact on saving lives in terms of care.

[0117] Furthermore, by using the ultrasonic system disclosed herein, the load on the resources of the care facility is reduced as compared to the use of conventional ultrasonic systems. This advantage is because the use of the ultrasonic system disclosed herein does not require sending the patient to another imaging department such as a radiology department, and can result in a successful diagnosis of PTX using only the ultrasonic system. In contrast, since conventional ultrasonic systems cannot properly diagnose PTX, as described above, they may require the use of additional imaging and, as a result, facilities with a higher resource load in the care facility than the ultrasonic systems disclosed herein. Therefore, the ultrasonic system disclosed herein can operate the care facility more efficiently compared to conventional ultrasonic systems, thereby providing better patient care.

[0118] Furthermore, the ultrasonic system disclosed herein operates faster than conventional ultrasonic systems that do not perform automated detection of lung sliding to assist in the diagnosis of PTX, so the operator can use the ultrasonic system disclosed herein to perform more complex ultrasonic examinations within a given time compared to conventional ultrasonic systems. Therefore, patients can receive better care using the ultrasonic system disclosed herein as compared to conventional ultrasonic systems.

[0119] There are some examples of the embodiments disclosed herein.

[0120] Example 1 is a method for lung sliding determination executed by a computing device, The method includes receiving one or more B-mode ultrasonic images including a pleural line, Generating a feature list from the one or more B-mode ultrasound images, wherein the feature list indicates at least one of the features of the pleural line, generating the feature list, and Generating the likelihood of the lung sliding using a neural network that is at least partially executed in the hardware of the computing device and configured to process the B-mode ultrasound images of the feature list and the one or more B-mode ultrasound images, A method comprising.

[0121] Example 2, wherein generating the likelihood of the lung sliding is based on the B-mode ultrasound image having a quality level exceeding a threshold quality level automatically and without user intervention, and activating the neural network to process the feature list and the B-mode ultrasound image. The method according to Example 1.

[0122] Example 3 further comprises determining a region of interest in the B-mode ultrasound image based on the position of the pleural line in the B-mode ultrasound image, Generating the likelihood of the lung sliding is based on the pixels of the B-mode ultrasound image included in the region of interest and not based on additional pixels of the B-mode ultrasound image not included in the region of interest. The method according to Example 1.

[0123] Example 4, wherein the position indicates the distance from the end of the pleural line to the end of the B-mode ultrasound image, Determining the region of interest is based on the distance. The method according to Example 3.

[0124] Example 5 further comprises generating additional likelihoods of the lung sliding based on additional B-mode ultrasound images of the one or more B-mode ultrasound images, The method according to Example 1.

[0125] Example 6, generating the additional probability of the Lang sliding is based on the feature list, and is the method described in Example 5.

[0126] Example 7 further includes generating an additional feature list from the additional B-mode ultrasound image, and generating the additional probability of the Lang sliding is based on the additional feature list, and is the method described in Example 5.

[0127] Example 8 includes merging the probability and the additional probability to form a combined probability of the Lang sliding, and displaying a representative of the combined probability of the Lang sliding on a user interface of the computing device, and is the method described in Example 5. and is the method described in Example 5.

[0128] Example 9 includes determining that an additional B-mode ultrasound image of the one or more B-mode ultrasound images is a redundant B-mode ultrasound image, and excluding the additional B-mode ultrasound image from the one or more B-mode ultrasound images to prevent the neural network from processing the additional B-mode ultrasound image, and further comprises, and is the method described in Example 1.

[0129] Example 10 is a method for Lang sliding determination executed by a computing device, wherein the method includes generating a B-mode ultrasound image, determining an instruction for improving the quality of the B-mode ultrasound image, and displaying the instruction on a user interface of the computing device, Generating additional B-mode ultrasound images based on user adjustments executed based on the command, and Generating the likelihood of the lung sliding, at least partially executed in the hardware of the computing device, using a neural network based on one or more of the additional B-mode ultrasound images, A method comprising.

[0130] Example 11 is that generating the likelihood of the lung sliding comprises activating the neural network based on one or more of the additional B-mode ultrasound images having a quality level exceeding a threshold quality level automatically and without user interference, The method according to Example 10.

[0131] Example 12 is that the B-mode ultrasound image includes a pleural line, The quality of the B-mode ultrasound image is based on the position of the pleural line in the B-mode ultrasound image, The method according to Example 10.

[0132] Example 13 is that one or more of the additional B-mode ultrasound images include a plurality of B-mode ultrasound images, Determining redundant B-mode ultrasound images among the plurality of B-mode ultrasound images, and Preventing the neural network from the generating the likelihood of the lung sliding by processing data determined from one or more of the redundant B-mode ultrasound images, the one or more redundant B-mode ultrasound images Excluding, Further comprising, The method according to Example 10.

[0133] Example 14 further comprises determining a region of interest in one or more of the additional B-mode ultrasound images, Generating the probability of the Langs sliding is based on the pixels of the one or more additional B-mode ultrasound images included in the region of interest and not based on additional pixels of the one or more additional B-mode ultrasound images not included in the region of interest. The method according to Example 10.

[0134] In Example 15, determining the region of interest is based on the pleural line in the one or more additional B-mode ultrasound images. The method according to Example 14.

[0135] In Example 16, the command Moving the ultrasound probe, Adjusting imaging parameters, and A proposal for selecting the neural network from a list of neural networks valid on the computing device including at least one of the guidance The method according to Example 10.

[0136] Example 17 is an ultrasound system for Langs sliding determination, The system A memory for maintaining ultrasound images and medical worksheets, A neural network that is at least partially executed in the hardware of the ultrasound system based on one or more ultrasound images and generates the probability of the Langs sliding, and A processing system that automatically and without user intervention presets columns of the medical worksheet using an indicator of the Langs sliding according to the probability generated by the neural network comprising An ultrasound system.

[0137] Example 18 further includes a display screen for performing the display. The medical worksheet is the column preset using the index of the Langs sliding, and the requirement for user confirmation in the index of the Langs sliding, including the ultrasonic system described in Example 17.

[0138] Example 19 further includes an ultrasonic probe having a position sensor configured to generate position data, the processing system is configured to determine the column of the medical worksheet based on the position data, which is the ultrasonic system described in Example 17.

[0139] Example 20 further includes an ultrasonic probe having a position sensor configured to generate position data, the processing system is configured to show, on the medical worksheet, a lung region scanned using the ultrasonic probe based on the position data, which is the ultrasonic system described in Example 17.

[0140] All of the methods and operations disclosed herein can be implemented and fully automated by a computer system. In some cases, the computer system includes multiple separate computers or computing devices (e.g., physical servers, workstations, storage arrays, cloud computing resources, etc.) that communicate and interoperate on a network to implement the disclosed functions. Each such computing device typically includes a processor (or multiple processors) that executes program instructions, or modules stored in a memory or other non-transitory computer-readable storage medium or device (e.g., solid-state storage device, disk drive, etc.). The various functions disclosed herein can be incorporated into such program instructions and can also be implemented in a circuit-special application of a computer system (e.g., ASICs or FPGAs). If a computer system includes multiple computing devices, these devices can be located in the same place, but they don't have to be. The disclosed methods and the results of operations can be permanently stored by converting from different states in a physical storage device such as a solid-state memory chip or a magnetic disk. In some embodiments, the computing device can be a cloud-based computing system where processing resources are shared by multiple separate vendors or other users.

[0141] Depending on the function of any of the embodiments, operations, events, or processes or algorithms described herein can be implemented in a different sequence, (e.g., not all of the described operations or events are necessary for the implementation of the algorithm), added, combined, or completely omitted. Furthermore, in certain embodiments, operations, or events, instead of being sequential, can be implemented simultaneously, for example, on a multi-threaded process, interrupt process, or multiple processors or processor cores, or other parallel architectures.

[0142] The various example logical blocks, modules, routines, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware (e.g., ASICs or FPGA devices), computer software operating on computer hardware, or a combination of both. Furthermore, the described logical blocks and modules that serve as various examples related to the embodiments disclosed herein can be executed or implemented by a machine device such as a processor device, a digital signal processing device (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other, discrete gate logic or transistor logic, individual hardware components, or any combination of these designed to perform the functions described herein. The processor device can be a microprocessor, however, in the alternative, the processor device can be a controller, a microcontroller, or a state machine device, a similar combination, or a similar device, etc. The processor device can include an electrical circuit configured to process computer-executable instructions. In another embodiment, the processor device includes an FPGA or other programmable device that performs logical operations without processing computer-executable instructions. The processor device can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors associated with a DSP core, or any other such configuration. Although mainly described herein with respect to digital technology, the processor device can also mainly include analog components. For example, some or all of the rendering techniques described herein can be implemented in an analog circuit or an analog and digital hybrid circuit. The computing environment can include any type of computer system, including, but not limited to, a microprocessor-based computer system, a mainframe computer, a digital signal processing device, a portable computing device, a device controller, or a computing engine within a household appliance.

[0143] The elements of a method, process, routine, or algorithm described in connection with the embodiments disclosed herein can be implemented directly in hardware, in software modules executed by a processor device, or in a combination of the two. A software module can reside in any form of RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or other form of non-transitory computer-readable storage medium. An exemplary storage medium can be coupled to the processor device such that the processor device can read information from, and write information to, the storage medium. Alternatively, the storage medium can be integrated into the processor device. The processor device and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. Alternatively, the processor device and the storage medium can reside as separate components in a user terminal.

[0144] Conditional language used herein, particularly, "can", "could", "might", "may", "e.g.", and the like, unless specifically stated otherwise, or understood otherwise from the context of use, is generally intended to convey that while one embodiment includes a particular feature, element, or step, another embodiment does not include that feature, element, or step. Accordingly, such conditional language is generally not intended to imply that a feature, element, or step is required in any way for one or more embodiments, nor is it intended to imply that one or more embodiments must include logic for determining whether these features, elements, or steps are included, or are to be performed in any particular embodiment, with or without other inputs or stimuli. The terms "comprising," "including," "having," and the like are synonymous and can be used interchangeably in an inclusive and open-ended manner, without excluding additional elements, features, acts, etc. Also, the word "or" is used in its inclusive sense (and not in its exclusive sense), so that when used, for example, to connect a list of elements, the word "or" means one, some, or all of the elements in the list.

[0145] Disjunctive language such as the phrase "at least one of X, Y, or Z" is generally understood from the context as being used to provide items, terms, etc., and unless specifically stated otherwise, X, Y, or Z can also be any combination thereof (e.g., X, Y, or Z). Accordingly, such disjunctive language is generally not intended to, nor should it be construed to, imply that an embodiment requires one or more of X, one or more of Y, and one or more of Z, respectively.

[0146] While the foregoing detailed description has shown, described, and pointed out novel features as applicable to various embodiments, it will be understood that various omissions, substitutions, and alterations in the form and detail of the devices or the algorithms shown can be made without departing from the spirit of the disclosure. As can be appreciated, some embodiments described herein can be incorporated into forms that do not provide all of the features and benefits disclosed herein, as some features can be used or implemented separately from others. The scope of certain embodiments disclosed herein is shown rather by the appended claims than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are embraced within their scope.

Claims

1. A method for determining lung sliding by a computing device, comprising: The method includes: Receiving one or more B-mode ultrasound images including a pleural line; Generating a feature list from the one or more B-mode ultrasound images, the feature list indicating at least one of the features of the pleural line, generating the feature list; and Generating a probability of lung sliding using a neural network that is at least partially executed in the hardware of the computing device and configured to process the feature list and the B-mode ultrasound images of the one or more B-mode ultrasound images. A method comprising the above.

2. The generating of the probability of lung sliding includes activating the neural network to process the feature list and the B-mode ultrasound images based on the B-mode ultrasound images having a quality level exceeding a threshold quality level automatically and without user intervention. The method according to claim 1.

3. Further comprising determining a region of interest in the B-mode ultrasound image based on the position of the pleural line in the B-mode ultrasound image; The generating of the probability of lung sliding is based on the pixels of the B-mode ultrasound image included in the region of interest and not based on additional pixels of the B-mode ultrasound image not included in the region of interest. The method according to claim 1.

4. The position indicates the distance from the end of the pleural line to the end of the B-mode ultrasound image; Determining the region of interest is based on the distance. The method according to claim 3.

5. Further comprising generating an additional probability of lung sliding based on additional B-mode ultrasound images of the one or more B-mode ultrasound images. The method according to claim 1.

6. The generating of the additional probability of lung sliding is based on the feature list. The method according to claim 5.

7. Further comprising generating an additional feature list from the additional B-mode ultrasound images; The generating of the additional probability of lung sliding is based on the additional feature list. The method according to claim 5.

8. merging the probability and the additional probability to form the combined probability of the Langs sliding; displaying, on a user interface of the computing device, a representative of the combined probability of the Langs sliding; further comprising; The method according to claim 5.

9. determining that an additional B-mode ultrasound image of the one or more B-mode ultrasound images is a redundant B-mode ultrasound image; excluding the additional B-mode ultrasound image from the one or more B-mode ultrasound images to prevent the neural network from processing the additional B-mode ultrasound image; further comprising; The method according to claim 1.

10. A method for Langs sliding determination executed by a computing device, the method comprising: the method comprising: generating a B-mode ultrasound image; determining an instruction for improving the quality of the B-mode ultrasound image; displaying the instruction on a user interface of the computing device; generating an additional B-mode ultrasound image based on user adjustment executed based on the instruction; and generating a probability of the Langs sliding using a neural network based on at least one of the additional B-mode ultrasound images, at least partially executed in hardware of the computing device; A method comprising.

11. The generating the probability of the Langs sliding includes activating the neural network based on the one or more additional B-mode ultrasound images having a quality level exceeding a threshold quality level automatically and without user interference; The method according to claim 10.

12. the B-mode ultrasound image includes a pleural line; the quality of the B-mode ultrasound image is based on a position of the pleural line in the B-mode ultrasound image; The method according to claim 10.

13. the one or more additional B-mode ultrasound images include a plurality of B-mode ultrasound images; determining redundant B-mode ultrasound images of the plurality of B-mode ultrasound images; Preventing the neural network by processing data determined from one or more of the redundant B-mode ultrasound images, thereby generating the probability of the lung sliding from the one or more redundant B-mode ultrasound images excluding; further comprising; The method according to claim 10.

14. further comprising determining a region of interest in the one or more additional B-mode ultrasound images, generating the probability of the lung sliding is based on pixels of the one or more additional B-mode ultrasound images included in the region of interest and not based on additional pixels of the one or more additional B-mode ultrasound images not included in the region of interest, The method according to claim 10.

15. determining the region of interest is based on a pleural line in the one or more additional B-mode ultrasound images, The method according to claim 14.

16. The instructions are moving an ultrasound probe, adjusting imaging parameters, and proposing to select the neural network from a list of neural networks available on the computing device including at least one of the guidance of, The method according to claim 10.

17. An ultrasound system for lung sliding determination, comprising the system is a memory for maintaining ultrasound images and a medical worksheet, a neural network that is at least partially executed in the hardware of the ultrasound system based on one or more ultrasound images and generates a probability of the lung sliding, and a processing system that automatically and without user intervention presets columns of the medical worksheet using an indicator of the lung sliding based on the probability generated by the neural network comprising, an ultrasound system.

18. further comprising a display screen configured to perform displaying, the medical worksheet is the columns preset using the indicator of the lung sliding, and a request for user confirmation in the indicator of the lung sliding, including, The ultrasound system according to claim 17.

19. further comprising an ultrasound probe having a position sensor configured to generate position data, The processing system is configured to determine the column of the medical worksheet based on the position data. The ultrasonic system according to claim 17. **Claim 20** The ultrasonic probe further includes a position sensor configured to generate position data. The processing system is configured to indicate, on the medical worksheet, a lung region scanned using the ultrasonic probe based on the position data. The ultrasonic system according to claim 17.

Images