Guided lung coverage and automated detection using ultrasound equipment

Abstract

The present invention discloses a system and method for guided lung coverage and automatic detection using an ultrasound device. The method for guided coverage and automatic pathology detection of a subject includes positioning an ultrasound probe on an area of the subject's body to be imaged. The method includes capturing a video of the subject and processing the video to generate a torso image of the subject and identifying the position of the ultrasound probe on the subject's body. The method includes aligning the video with an anatomical atlas to generate a mask of the area of the subject's body to be imaged and superimposing the mask on the torso image, the area including multiple sub-regions of the subject's body. The method also includes displaying an indicator corresponding to the position of each sub-region in the multiple sub-regions on the torso image. The method also includes displaying the relative position of the ultrasound probe relative to the indicator, the indicator corresponding to the position of each sub-region in the multiple sub-regions of the subject's body to be imaged.

Description

Technical Field

[0001] The present disclosure relates generally to improved medical imaging systems and methods, and more particularly to systems and methods for guided lung coverage and automatic detection of pathologies by adjusting the position of an ultrasound probe on a subject's body. Background Art

[0002] Various medical imaging systems and methods are used to obtain images of affected areas of a subject for use in diagnosing medical conditions. Ultrasound imaging is a known medical imaging technique that is used to image various body parts such as joints, muscle blood vessels, and pregnant women (referred to as obstetric ultrasound). Ultrasound imaging offers several advantages over other imaging techniques because ultrasound is a real-time imaging technique that provides a real-time image stream. Ultrasound equipment is commercially available in a variety of configurations, and portable ultrasound equipment is used to capture relatively large areas of a subject's anatomy, such as the uterus, lower abdomen, and lungs.

[0003] Ultrasound imaging involves generating and transmitting ultrasonic waves toward the part of a subject's body to be imaged, and receiving reflected waves from the subject's body. Ultrasound imaging equipment consists of a probe that can be positioned on the subject's skin over the pathological part to be imaged. The probe transmits ultrasonic waves into the subject's body and captures the reflected waves to generate an image of the subject's pathology.

[0004] Known image processing techniques are used to create user-viewable ultrasound images. However, accurately imaging a subject using a portable ultrasound device requires a skilled operator. Any operator error will result in the image containing excessive interference or noise. Therefore, the operator's training and experience in operating the ultrasound imaging device will affect the quality of the ultrasound image. Due to the operator's experience in acquiring images of certain parts of the body, the operator may be proficient in acquiring high-quality images of that part of the body, but the operator may not be proficient in acquiring high-quality images of the entire subject's body. This may be due to a lack of imaging experience or a lack of knowledge of the entire body's anatomy. In addition, placing the ultrasound probe at the appropriate part of the body can affect image quality. The subject's physical parameters, such as obesity, bone density, height, and chest or abdominal size, can make it challenging for the operator to correctly position the ultrasound probe to obtain a complete image of the region of interest. Any incorrect placement of the ultrasound probe will result in incomplete imaging of the region of interest.

[0005] Coronavirus disease (COVID-19), as defined by the World Health Organization (WHO), is an infectious disease caused by a newly discovered coronavirus. One of the symptoms frequently observed in subjects infected with COVID-19 is lung infection, and in severe cases, pneumonia. Frequent imaging is recommended to monitor the progression of COVID-19 infection. Therefore, it is important to obtain higher-quality ultrasound images of subjects, which will provide valuable information about the progression of COVID-19 disease. Rapid response to epidemics has become and will remain a pressing need in the future. Ultrasound allows for rapid diagnosis of lung conditions, which is particularly important in the current COVID-19 pandemic. Due to the severe shortage of trained medical personnel, novice users such as paramedics and nurses should be able to quickly assess lung conditions using ultrasound equipment. Lack of scanning expertise in operating ultrasound equipment will result in the production of poor-quality images, leading to a poor assessment of the subject's condition. This could lead to the loss of many precious lives.

[0006] With the outbreak of the Covid-19 pandemic, global healthcare resources have been stretched to their limits. Rapid, large-scale screening will require the services of paramedics and less-skilled healthcare workers. Ultrasound has been shown to detect pleural effusions and lung consolidation, but it requires considerable operator skill to manipulate the probe. Furthermore, Covid-19 exhibits certain locational patterns that must be accounted for during the triage process. Covid-19 manifestations are often multifocal and peripheral, often involving the lower lobes. Because ultrasound has a very limited field of view (FoV), it is impossible to know whether all Covid-19-affected areas, or hotspots, within a patient's lungs have been captured during a lung exam.

[0007] What is needed are systems and methods for providing automated guidance to a scanner to properly position and operate an ultrasound probe to scan portions of a subject's lungs and generate images that will aid in tracking COVID-19 disease progression. Summary of the Invention

[0008] This summary introduces concepts that are described in greater detail in the detailed description. It is not intended to identify essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Its sole purpose is to present the concepts in a simplified form as a prelude to the more detailed description that is presented later.

[0009] According to one aspect of the present disclosure, a method for guided overlay and automatic detection of pathology on a subject is disclosed. The method includes positioning an ultrasound probe on an area of the subject's body to be imaged. The method also includes capturing a video of the subject's body and processing the video to generate a torso image of the subject's body and identifying the position of the ultrasound probe on the subject's body. The method also includes aligning the video with an anatomical atlas to generate a mask of the area of the subject's body to be imaged and superimposing the mask on the torso image, the area including multiple sub-regions of the subject's body. The method also includes displaying an indicator corresponding to the position of each of the multiple sub-regions on the torso image. The method also includes displaying the relative position of the ultrasound probe relative to the indicator, the indicator corresponding to the position of each of the multiple sub-regions of the subject's body to be imaged.

[0010] According to one aspect of the present disclosure, a system for guided overlay and automatic pathology detection of a subject's body is disclosed. The system includes a portable device comprising a camera and a display configured to capture a video of the subject's body. The system also includes an ultrasound probe positioned above an area of the subject's body to be imaged and connected to the portable device. The system also includes a computer system connected to the portable device and configured to receive multiple ultrasound images and videos of the subject's body. The computer system includes a processor and a memory connected to the processor. The computer system also includes at least one artificial intelligence module, which is deployed on the memory and configured to generate a torso image of the subject's body. The computer system also includes an atlas module, which is deployed on the memory and configured to generate a mask of the area of the subject's body to be imaged and superimpose the mask on the torso image of the subject's body, the area including multiple sub-areas. The computer system is further configured to display an indicator corresponding to a position of each of the plurality of sub-regions on the torso image, and to display a relative position of the ultrasound probe relative to the indicator corresponding to the position of each of the plurality of sub-regions on the subject's body. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 Shown is an ultrasound image of a subject with a pneumonia-like condition according to one aspect of the present disclosure.

[0012] Figure 2 A system for guided overlay and automatic detection of pathology on a subject according to one aspect of the present disclosure is shown.

[0013] Figure 3 An image of a subject obtained using a portable device according to one aspect of the present disclosure is shown.

[0014] Figure 4 An atlas of receiving a torso image and using external landmarks is shown according to one aspect of the present disclosure.

[0015] Figure 5 Shown are a plurality of grid lines generated by an atlas module over a mask of a lower lung region according to one aspect of the present disclosure.

[0016] Figure 6 A method for guided overlay and automatic detection of pathology on a subject according to one aspect of the present disclosure is shown.

[0017] Figure 7 A method for analyzing accurate positioning of an ultrasound probe over a subject's body according to one aspect of the present disclosure is shown.

[0018] Figure 8 Shown is a torso image displayed to an operator for imaging according to one aspect of the present disclosure.

[0019] Figure 9 Shown is a torso image displayed to an operator for guided lung coverage and automatic detection according to one aspect of the present disclosure. DETAILED DESCRIPTION

[0020] In the following specification and claims, reference will be made to several terms that shall be defined to have the following meanings.

[0021] The singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise.

[0022] As used herein, the term "non-transitory computer-readable medium" is intended to mean any tangible computer-based device implemented in any method or technology for short-term and long-term storage of information, such as computer-readable instructions, data structures, program modules and submodules, or other data in any device. Thus, the methods described herein may be encoded as executable instructions embodied in a tangible, non-transitory computer-readable medium, including but not limited to a storage device and / or a memory device. When executed by a processor, such instructions cause the processor to perform at least a portion of the methods described herein. Furthermore, as used herein, the term "non-transitory computer-readable medium" includes all tangible computer-readable media, including but not limited to non-transitory computer storage devices, including but not limited to volatile and non-volatile media, and removable and non-removable media, such as firmware, physical and virtual storage, CD-ROMs, DVDs, and any other digital source (such as a network or the Internet), as well as digital devices yet to be developed, with the sole exception of transient propagation signals.

[0023] As used herein, the terms "software" and "firmware" are interchangeable and include any computer program stored in memory for execution by devices including, but not limited to, mobile devices, clusters, personal computers, workstations, clients, and servers.

[0024] As used herein, the term "computer" and related terms (e.g., "computing device," "computer system," "processor," "controller") are not limited to the integrated circuits known in the art as computers, but refer broadly to at least one microcontroller, microcomputer, programmable logic controller (PLC), application specific integrated circuit, and other programmable circuits, and these terms are used interchangeably herein.

[0025] As used herein throughout the specification and claims, approximating language may be applied to modify any quantitative representation that can be permissibly varied without resulting in a change in the basic function to which it is related. Thus, a value modified by one or more terms such as "about" and "substantially" is not limited to the precise value specified. In at least some cases, approximating language may correspond to the precision of an instrument used to measure the value. Here and throughout the specification and claims, range limitations may be combined and / or interchanged, and such ranges may be identified and include all subranges contained therein unless context or language indicates otherwise.

[0026] In one aspect of the present disclosure, a method for guided overlay and automatic detection of pathology in a subject is disclosed. The method includes positioning an ultrasound probe over an area of the subject's body to be imaged. The method also includes capturing a video of the subject's body and processing the video to generate a torso image of the subject's body and identifying the position of the ultrasound probe on the subject's body. The method also includes aligning the video with an anatomical atlas to generate a mask of the area of the subject's body to be imaged and superimposing the mask over the torso image, the area including multiple sub-regions of the subject's body. The method also includes displaying an indicator corresponding to the position of each of the multiple sub-regions on the torso image. The method also includes displaying the relative position of the ultrasound probe relative to the indicator, the indicator corresponding to the position of each of the multiple sub-regions of the subject's body to be imaged.

[0027] In another aspect of the present disclosure, a system for guided overlay and automatic pathology detection of a subject is disclosed. The system includes a portable device comprising a camera and a display configured to capture a video of the subject's body. The system also includes an ultrasound probe positioned above an area of the subject's body to be imaged and connected to the portable device. The system also includes a computer system connected to the portable device and configured to receive multiple ultrasound images and videos of the subject's body. The computer system includes a processor, a memory connected to the processor, and at least one artificial intelligence module disposed on the memory and configured to generate a torso image of the subject's body. The computer system also includes an atlas module disposed on the memory and configured to generate a mask of the area of the subject's body to be imaged and superimpose the mask on the torso image of the subject's body, the area including multiple sub-areas. The computer system is configured to display an indicator corresponding to the position of each of the multiple sub-regions on the torso image, and to display a relative position of the ultrasound probe relative to the indicator above the display screen of the portable device, the indicator corresponding to the position of each of the multiple sub-regions on the subject's body.

[0028] Embodiments of the present disclosure will now be described by way of example with reference to the accompanying drawings, in which Figure 1 An ultrasound image (10) of a subject (not shown) with symptoms of pneumonia (11) and air bronchograms (12) is shown. In Covid-19 cases, pneumonia (11) is one of the important signs of severe spread of infection. These symptoms are observed in the peripheral and lower regions of the lungs. When an ultrasound image (10) of the lungs is captured, the pneumonia symptoms may appear as texture changes in the ultrasound image (10). Capturing the ultrasound image (10) without the expertise required to obtain full coverage of the lungs may result in missing some affected areas. This may lead to anomalies in the diagnosis of the subject's condition.

[0029] In one aspect of the present disclosure, Figure 2A system (100) for guided overlay and automatic detection of pathology on a subject (101) is shown. The system (100) includes a portable device (102) used by an operator to capture a video stream or image of the subject (101). The portable device (102) can be a mobile phone having a camera for capturing a video stream or image of the subject (101) and including a display screen. A computer system (103) can be connected to the portable device (102), and the video or real-time stream from the portable device (102) can be transmitted to the computing device (103) for further processing. An ultrasound probe (104) can be configured to obtain an ultrasound image of the subject (101), and the ultrasound probe (104) can be connected to the portable device (102).

[0030] The computing device (103) may include a processor (111), a memory (112), and at least one artificial intelligence module (113) disposed on the memory (112). The artificial intelligence module (113) may be a deep learning module that can be trained to compare a video stream or image acquired by a mobile camera with stored images of a subject's body of different shapes, and to identify an image that best matches the actual shape of the subject (101). The artificial intelligence module (113) may be configured to recognize and generate an image of the subject's (101) torso and landmarks on the image, such as reference Figure 3 and Figure 4 Explained in detail.

[0031] The computing device (103) may include an atlas module (114) configured to register an image or video stream from a mobile camera. The atlas module (114) registers to ensure external scale alignment and may provide grid lines on the received image. The atlas module (114) may be configured to generate a lung mask including grid lines and overlay the mask on the torso image to indicate a plurality of subregions of interest (560) for imaging. The atlas module (114) output including the plurality of subregions (560) may be displayed to an operator via a mobile phone to indicate the subregions (560) to be imaged.

[0032] In addition, the position of the ultrasound probe (104) can be obtained by the mobile camera and indicated to the operator through the mobile device (102) screen. When the ultrasound probe (104) is correctly positioned above at least one of the sub-regions (560) indicated by the atlas module (114), an audiovisual signal can be given to the operator to perform the imaging procedure. Alternatively, when the position of the ultrasound probe (104) above the body of the subject (101) does not match at least one of the sub-regions (560) indicated by the atlas module (114), another audiovisual signal or instruction can be provided to the operator to change the position of the ultrasound probe (104) to the identified position. Continuous audiovisual instructions can be provided to the operator to guide the operator to accurately position the ultrasound probe (104) above at least one of the marked sub-regions (560). When the operator positions the ultrasound probe over at least one of the subregions (560) and performs imaging at the indicated subregion (560) of the subject (101), a detailed ultrasound image of the pathology of the subject's lung can be obtained.

[0033] In one aspect of the present disclosure, in order to more quickly detect the COVID-19 condition that a subject (101) is developing, an ultrasound image of the subject (101) can be compared with similar images to identify the severity of the subject's medical condition. Several ultrasound streams including images of various sub-regions (560) of the lungs from different subjects (101) can be used to train an artificial intelligence module (113). In one example, the artificial intelligence module (ultrasound AI) (113) can be configured to analyze pathology to identify the spread of COVID-19 on various sub-regions (560) of the lungs. The artificial intelligence module (113) can be a single module or a collection of several single modules that are trained to perform a function, and include, for example, a deep learning (DL) module, a convolutional neural network, a supervised artificial intelligence module, a semi-supervised artificial intelligence module, or an unsupervised artificial intelligence module. The ultrasound AI (113) can be used with a mobile camera to detect the position of the probe (104) and perform imaging. If the ultrasound probe (104) is placed in an incorrect position, the ultrasound AI (113) can be configured to send a signal indicating the incorrect positioning of the ultrasound probe (104). The ultrasound probe (104) can be moved to the correct position to perform guided lung coverage and automatic pathology detection using the artificial intelligence module (113). Guided lung coverage and automatic pathology detection can be performed by the operator for all sub-regions (560) indicated on the display screen.

[0034] Deep learning modules, machine learning modules, and artificial intelligence networks

[0035] The terms deep learning (DL), machine learning (ML), and artificial intelligence (AI) are related and often used interchangeably. However, AI is a broader technology that encompasses intelligent machines with the ability to think, while machine learning involves learning from data.

[0036] Deep learning is a type of machine learning technique that uses representation learning methods, allowing a machine to be given raw data and determine the representation needed to classify the data. Deep learning uses the backpropagation algorithm to detect structure in a dataset. Deep learning machines can utilize a variety of multi-layer architectures and algorithms. For example, while machine learning involves identifying features to train a network, deep learning processes raw data to identify interesting features without external identification.

[0037] Deep learning in a neural network environment involves numerous interconnected nodes called neurons. Input neurons, activated by external sources, activate other neurons based on connections to those neurons, which are controlled by the machine's operating conditions. Neural networks function in a specific way based on their own sequences. Learning improves the machine's output and, more broadly, the connections between neurons in the network, causing the neural network to function in the desired manner.

[0038] Deep learning using convolutional neural networks uses convolutional filters to segment data to locate and identify learned observable features in the data. Each filter, or layer, of the CNN architecture transforms the input data to increase its selectivity and invariance. This abstraction of the data allows the machine to focus on the features it is trying to classify in the data and ignore irrelevant background information.

[0039] Deep learning operates on the understanding that many datasets consist of high-level features, which in turn consist of lower-level features. For example, when examining an image, rather than looking for objects, it's more efficient to look for edges, which form motifs, which form parts, and which form the objects being sought. This hierarchy of features can be seen in many different forms of data, such as speech and text.

[0040] The learned observable features include the objects and quantifiable regularities that the machine learned during supervised learning. A machine that is provided with a large set of data that it has effectively classified is better equipped to distinguish and extract features that are relevant to the successful classification of new data.

[0041] A deep learning machine using transfer learning can correctly link data features to certain classifications confirmed by human experts. Conversely, the same machine can update the system used for classification when a human expert reports a classification error. For example, settings and / or other configuration information can be guided by the use of learned settings and / or other configuration information, and the number of changes and / or other possibilities for settings and / or other configuration information for a given situation can be reduced as the system is used more times (e.g., repeatedly and / or by multiple users).

[0042] For example, an exemplary deep learning neural network can be trained using an expert classification dataset. This dataset builds the neural network, and this becomes the supervised learning phase. During the supervised learning phase, the neural network can be tested to see if it has achieved the desired behavior.

[0043] Once the desired neural network behavior has been achieved (e.g., the machine has been trained to operate according to specified thresholds, etc.), the machine can be deployed for use (e.g., using "real" data to test the machine, etc.). During operation, the neural network classification can be confirmed or rejected (e.g., by an expert user, an expert system, a reference database, etc.) to continue improving the neural network behavior. The exemplary neural network is then in a state of transfer learning, as the classification criteria that determine the neural network behavior are updated based on ongoing interactions. In some examples, the neural network can provide direct feedback to another process. In some examples, the data output by the neural network is buffered (e.g., via the cloud, etc.) and validated before being provided to another process.

[0044] Deep learning machines using convolutional neural networks (CNNs) can be used for image analysis. CNN analysis stages can be used for facial recognition in natural images, computer-aided diagnosis (CAD), and more.

[0045] Interpreting medical images, regardless of quality, is a relatively recent development. Medical images are largely interpreted by physicians, but these interpretations can be subjective, influenced by physician experience and / or fatigue in the field. Image analysis via machine learning can support healthcare practitioners' workflows.

[0046] For example, deep learning machines can provide computer-aided detection support to improve image analysis in terms of image quality and classification. However, deep learning machines used in the medical field often face challenges, resulting in many misclassifications. For example, deep learning machines must overcome small training datasets and require repeated adjustments.

[0047] For example, deep learning machines can be used to determine the quality of medical images with minimal training. Semi-supervised and unsupervised deep learning machines can be used to quantitatively measure aspects of image quality. For example, deep learning machines can be used after an image has been acquired to determine whether the image quality is sufficient for diagnosis. Supervised deep learning machines can also be used for computer-aided diagnosis. For example, supervised learning can help reduce susceptibility to misclassification.

[0048] Deep learning machines can leverage transfer learning when interacting with physicians to offset the small datasets available for supervised training. These deep learning machines can improve their computer-aided diagnosis over time through training and transfer learning.

[0049] According to one aspect of the present disclosure, Figure 3 An image (300) of the torso of a subject (101) is shown generated using a deep learning (DL) module from images obtained using a camera of a portable device (102). The portable device (102) may be any known commercially available device having at least one camera that can capture video or images of the subject (101). The portable device (102) may be a handheld portable device or a fixed device that is configured to acquire, store, and transmit images of the subject to other devices via a wired or wireless network. According to one aspect of the present disclosure, the portable device (102) may be a mobile phone having a camera that can operate simultaneously with an ultrasound probe. During operation of the camera, the mobile phone can be held by an operator and a video stream or image of the subject's body can be captured. The mobile phone camera is capable of imaging a larger surface area of the subject's body than the ultrasound probe (104).

[0050] A video or image of a subject (101) captured by a mobile phone camera can be sent to and stored in a computer memory (112). The video or image of the subject (101) obtained by the mobile phone camera can be used to generate a torso image (310) of the subject (101) and identify the position of the ultrasound probe (104) on the subject's body (101). The torso image (300) can be a front view, a rear view, or a side view of the subject's body (101). A deep learning (hereinafter referred to as DL) module (113) can be used on the computer system (103) to perform body posture detection of the head (320) or the torso (310). The deep learning module (113) can be trained using different available body images to recognize the head (320) or the torso (310). The image obtained using the mobile phone camera is processed by the DL module (113) to generate a torso image (300) of the subject (101). The DL module may also be configured to perform body posture estimation. Body posture estimation includes using the DL module (113) to identify the alignment and orientation of various organs of the subject's body (101). In order to detect the position of specific organs and delineate different organs for imaging, posture detection based on the DL module (113) may provide landmarks (330) on the subject's body. The landmarks (330) may define the posture, orientation, and boundaries of various organs of the subject's body (101). The mobile phone may also be configured to receive an ultrasound image of the subject (101) obtained using an ultrasound probe (104) positioned above the subject's body.

[0051] According to one aspect of the present disclosure, Figure 4 Atlas-based segmentation (400) and registration are shown. Atlas-based segmentation of a torso image (410) of a subject's body (101) is a medical imaging technique used to delineate the contours of the subject's organs (e.g., hands (411), head (412), lungs (413), and abdomen (414)) relative to each other and save the time required by an expert to delineate. An atlas module (114) can be employed on a computer system for segmentation and delineation. Images obtained using a mobile phone camera can be processed by a DL module (113) for body posture detection and sent to the atlas module (114) for segmentation and registration. The atlas module (114) uses landmarks (420) detected externally by the DL module (113) to segment the image and register the image with the atlas module (114). The atlas module (114) can also be configured to generate a mask of the organ that can be superimposed on the torso image and displayed to the operator on the mobile phone screen.

[0052] According to one aspect of the present disclosure, Figure 4As shown, the atlas (400) receives an image of the torso (410) and uses the external landmarks (420) detected by the DL module (113) to register the image with the atlas module (114). Atlas registration ensures that the overall pose and external scale are aligned.

[0053] According to one aspect of the present disclosure, Figure 5 As shown, the atlas module (114) can be configured to provide a user-viewable image (500) of the torso of the subject's body (101). The atlas module (114) can be configured to generate a lung mask (520) that can be superimposed on the torso image (510) of the subject's body (101) and displayed on a mobile screen. The mask image (520) is an image in which some of the pixel intensity values are zero and others are non-zero. If the pixel intensity value in the mask image (520) is zero, the pixel intensity of the resulting masked image will be set to the background value. The zero mask value represents a background area of the image that is unimportant and does not contain any meaningful information. The pixels inside the mask (520) that contain meaningful information do not change their pixel values, and they maintain the same intensity as the input image.

[0054] A novice operator can see a video stream of the subject's body (101) and a torso image (510), as well as the position of the ultrasound probe (104) on the subject's body (101) which can be viewed in a window on the mobile phone screen (102). The mobile phone screen displays a lung mask (520) superimposed on the torso image (510) by the atlas module (114) and the current position of the ultrasound probe (104) on the subject's body. In addition, the atlas module (114) can be configured to generate a plurality of grid lines (530, 540, 550) above the mask (520) and indicate the location of the lung sub-region (560) to be imaged. When the views of the subject's lungs, ultrasound probe placement, and grid lines (530, 540, 550) are available, the operator has all the basic views and controllable parameters that can be used to perform a complete imaging of the subject's lungs.

[0055] During the examination of various subjects (101), subjects (101) of various shapes, sizes and heights will be imaged. It is important to correctly identify the location of the subject's organs on the image of the subject's body and appropriately instruct the operator to image the subject (101). The artificial intelligence module (113) can be trained to identify the location of various organs on the subject's body and display the location of the organs to the operator. The artificial intelligence module (113) can be employed during the video acquisition phase (video AI) to identify the actual shape of the body and correlate it with the stored images to determine the location of the subject's organs. This will complement the atlas module (114) to more accurately depict the organs and grid lines (530, 540, 550).

[0056] The atlas module (114) can be configured to provide grid lines (530, 540, 550) above the mask (520) that more precisely define the lower lung region of the subject (101). According to one aspect of the present disclosure, the atlas module (114) can insert a parasternal line (530), an anterior axillary line (540), and a posterior axillary line (550) above the mask (520), which lines mark the lungs of the subject's body. These grid lines (530, 540, 550) can be displayed to the operator on a mobile phone screen. In addition, the grid lines (530, 540, 550) can divide the mask (120) into parts (1, 2, 3, 4) to more accurately define the locations of different sub-regions (520). The atlas module (114) can also divide the lung area in a suitable area for imaging. The atlas module (114) can be configured to mark key areas or sub-regions (560) in the lungs that require detailed imaging. In addition, the superimposed grid lines (530, 540, 550) can further divide the lung surface area to precisely define the location of the sub-region (560) that requires detailed imaging. For the convenience of the operator of the mobile device (102), the display screen can only show the sub-region (560) located within the grid lines (530, 540, 550). In addition, each of the sub-regions (560) can be shown by a suitable indicator at a position corresponding to the position of the sub-region (560). In one example, the indicator corresponding to the sub-region (560) can be a region (561, 562) shown in red. According to one aspect of the present disclosure, it may not be necessary to display the mask (520) to the operator, however, only the indicator (561, 562) corresponding to the location of the sub-region (560) can be displayed to the operator for imaging. The position of the ultrasound probe (104) can be adjusted to scan the sub-region (560) marked by the atlas module (114). When the ultrasound probe (104) is correctly positioned to scan the subregion (560) indicated by the atlas module (114), a signal regarding the accurate positioning of the ultrasound probe (104) may be sent to the operator. When the ultrasound probe (104) is not accurately positioned above the marked subregion (560) of the lung, the image may not clearly show the pathology of the subject (101). A signal indicating the inaccurate positioning of the ultrasound probe (104) may be sent to the operator. The computer system (103) may indicate the incorrect positioning of the ultrasound probe (104) to the operator and guide the operator to change the position of the ultrasound probe (104) to the marked subregion (560) to perform detailed imaging. In one example, if the ultrasound probe (104) is placed three centimeters (3 cm) to the right of the marked subregion (560), the operator may be guided to move the ultrasound probe (104) 3 cm to the left for imaging.Similarly, the operator can be guided to move the ultrasound probe (104) in a vertical, lateral, or diagonal direction for accurate positioning above the sub-region (560) or sub-region indicators (561, 562). For novice operators, the position of the ultrasound probe (104) on the subject's body is displayed and the position of the sub-region (560) to be imaged is indicated, guiding the operator to correctly position the ultrasound probe (104) above the sub-region (560). The image obtained by accurately positioning the ultrasound probe (104) above the sub-region (560) is clinically more relevant to diagnosing pneumonia-like symptoms in COVID-19 disease.

[0057] In addition, the accurate placement of the ultrasound probe (104) on the subject's body at the sub-region (560) indicated by the atlas module (114) needs to be continuously monitored. Once the ultrasound probe (104) is positioned at the correct position indicated by the atlas module (114), a correlation can be established between the image obtained by the ultrasound probe (104) and the position of the ultrasound probe (104). This correlation helps determine the location of the COVID-19 pathology. In addition, these ultrasound images can be stored for future reference, thereby analyzing the progression of the COVID-19 disease by comparing a new set of ultrasound images obtained in the future with the stored images.

[0058] In another aspect of the present disclosure, Figure 6 A method (600) for performing guided automatic detection of lung coverage and pathology is shown. The method (600) assists an operator of an ultrasound probe (104) in accurately positioning the ultrasound probe (104) over a subject's body and obtaining an ultrasound image of the subject's body (101). The method (600) includes positioning (610) the ultrasound probe (104) over the subject's body (101) for imaging. The method (600) also includes capturing (620) a video of the subject (101) and processing the video to generate an image of the subject's (101) torso and identify the position of the ultrasound probe (104) on the subject's body. Capturing (620) the video of the subject (101) includes using a portable device (102) to obtain a video stream or image of the subject's body. The portable device (102) may be a handheld mobile phone having a camera for capturing a video stream or image of the subject (101) and a display screen.

[0059] The method (600) also includes registering the video with an anatomical atlas (630) to generate a mask of a region of the subject's body. The mask may include multiple subregions (560) of the subject's body to be imaged, and the mask may be superimposed on the torso image. Registering the video (630) may include connecting the portable device (102) to the computer system (103) using a cable or a wireless network and sending a real-time video stream or image to the computer system (103). The anatomical atlas deployed on the computer system (103) may be configured to process the real-time stream or image provided by the portable device (102). The computer system (103) may include a deep learning (DL) module (113) that may be configured to perform body posture detection and generate a torso image of the subject (101) based on an input image received from the portable device (102). The atlas module (114) may be configured to generate a mask of the lungs and superimpose the mask on the torso image visible on the mobile screen. The method (600) further includes generating, by the atlas module (114), a grid of lines above the image of the torso of the subject (101) to identify regions of interest for imaging. The atlas module (114) may also be configured to mark a plurality of subregions (560) within the grid that more accurately define the precise location of the pathology for imaging. The method (600) further includes displaying (640) the relative position of the ultrasound probe (104) relative to the plurality of subregions (560) on the subject's body to be imaged.

[0060] like Figure 7 As shown, the method (600) also includes analyzing (650) the accuracy of the position of the ultrasound probe (104) over the subregion (560) to be imaged of the subject's body. The computer system (103) can be configured to identify the current position of the ultrasound probe (104) and compare it to the position of the subregion (560) marked by the atlas module (114). The method (600) also includes generating (660) a signal indicating an inaccurate position of the ultrasound probe (104) on the subject (101). The signal can be an audiovisual signal indicating the inaccurate position of the ultrasound probe (104) relative to the marked subregion (560). Repeated signals indicating the inaccurate position of the ultrasound probe (104) can be sent to the operator until the operator moves the probe to the correct position or subregion (560). The ultrasound probe (104) can be moved to an accurate position for the subregion (560) on the subject's body (101).

[0061] The method (600) further includes generating (670) a signal indicating the accurate positioning of the ultrasound probe (104) over the subregion (560) of the subject's body and performing a guided ultrasound scan of the subject's (101) pathology. Once the ultrasound probe (104) is correctly positioned at the marked subregion (560), the computer system (103) can instruct the operator to scan the subject. An ultrasound image or video stream of the lung subregion (560) with a desired image quality can be obtained to detect COVID-19 related lung conditions. The method (600) further includes using (680) the video stream or image obtained by accurately placing the ultrasound probe (104) to determine the lung area where COVID-19 and pneumonia pathology are present.

[0062] The method (600) may also include employing an artificial intelligence module (113) (video AI) by the computer system (103) to identify the actual shape of the body and correlate it with the stored images to determine the location of the subject's organs during video imaging by the mobile phone camera. This will complement the atlas module (114) to more accurately delineate the organs, generate torso images and grid lines. This will help to correctly map the subject's organs when subjects (101) of different shapes, sizes and heights are presented for scanning.

[0063] The method (600) may also include employing another artificial intelligence module (113) (ultrasound AI) through the computer system (103). The method (600) includes using the ultrasound AI in conjunction with the mobile phone camera to detect the position of the ultrasound probe (104) and perform a scan. If the ultrasound probe (104) is placed at an inaccurate position, the ultrasound AI (113) may be configured to generate and send a signal indicating the inaccurate positioning of the ultrasound probe (104). In order to more quickly detect a developing COVID-19 condition in the subject (101), the ultrasound image of the subject (101) may be compared with similar images to identify the severity of the medical condition. Several ultrasound videos or images of various sub-regions (560) of the lungs from different subjects (101) may be used to train the ultrasound AI module (113) to analyze pathology for COVID-19 detection.

[0064] According to one aspect of the present disclosure, Figure 8A torso image (800) is shown, which includes placing an ultrasound probe (904) over a subject's body (901) to an operator for guided coverage and automatic detection of pathology. Inaccurate positioning of the ultrasound probe (804) over the subject's body (801) can be indicated to the operator via the screen of a portable device (850). The portable device (850) can be a mobile phone. The operator can place the ultrasound probe (804) at any position (810) over the subject's body (801). Sub-areas (860) along with grid lines (820, 830, 840) can be displayed to the operator via the mobile phone (850) screen for imaging. According to one aspect of the present disclosure, if the ultrasound probe (804) is not accurately placed at the sub-region (860) indicated on the torso image (800) on the mobile phone (850), the AI module (113) is configured to generate an audiovisual signal (841) indicating the inaccurate positioning of the ultrasound probe (804) above the sub-region (860). In one example, the signal may be a red box indicating the inaccurate positioning of the ultrasound probe above the sub-region (860). Although red is used as an example, any other color scheme may be used for the visual display. In addition, using any other pattern of signal indication (851) is also within the scope of the present disclosure, including but not limited to a right-wrong symbol, a human thumb representation, a stop-start button for indicating the accuracy of the placement of the ultrasound probe (804). In addition, the AI module (113) may be configured to generate a text message (852) regarding the position of the ultrasound probe (804) and the direction in which the ultrasound probe (804) can be moved to accurately position the probe. A text message (852) may be displayed to the operator via the mobile phone (850) screen and will guide the operator to move the ultrasound probe (804) to the correct position.

[0065] According to one aspect of the present disclosure, Figure 9A torso image (900) is shown, which includes placing an ultrasound probe (904) above a subject's body (901) for guided coverage and automatic detection of pathology. The torso image (900) may include grid lines (920, 930, 930), and a sub-region (960) to be imaged may be located within the grid lines (920, 930, 940). When the ultrasound probe (904) is accurately positioned above the identified sub-region (960) above the subject's body (901), the AI module (113) may generate a signal indicating the accurate positioning of the ultrasound probe (904) above the sub-region (960). The accurate positioning of the ultrasound probe (904) above the sub-region (960) may be indicated to the operator by the signal via the screen of the mobile phone (950). The signal generated by the AI module (113) may be an audiovisual signal. In one example, the signal may be a green box (951) indicating the accurate positioning of the ultrasound probe (904) over the sub-region (960). Although the green color is used as an example, any other color scheme may be used for the visual display. In addition, the use of any other pattern of signal indications (951) is also within the scope of the present disclosure, including but not limited to a correct-wrong symbol, a human thumb representation, a stop-start button. In addition, the AI module (113) may be configured to generate a text message (952) indicating the accurate positioning of the ultrasound probe (904) and instructing the operator to perform imaging. The text message (952) may be displayed to the operator through the mobile phone (950) screen, and the operator will be guided to perform a scan for automatic detection of pathology.

[0066] The systems and methods of the present disclosure can provide the rapid diagnosis that is often required for COVID-19 infected subjects (101), even when the operator is relatively inexperienced. The systems and methods of the present disclosure can be used to detect not only pathologies within the lungs, but also other pathologies in the subject's body. Furthermore, the systems and methods of the present disclosure are cost-effective and may not require multiple sensors for detecting pathologies or for identifying the exact position of an ultrasound probe.

[0067] This written description uses examples to disclose the invention, including the best mode, and to enable any person skilled in the art to practice the invention, including making and using any computing system or systems and performing any included methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insignificant differences from the literal language of the claims.

Claims

1. A method for guided coverage and automatic pathology detection of a subject's body, the method comprising: positioning an ultrasound probe over the area of the subject's body to be imaged; capturing a video of the subject's body and processing the video to generate a torso image of the subject's body and identify a position of the ultrasound probe relative to the subject's body; registering the video with an anatomical atlas to generate a mask of the region of the subject's body to be imaged and superimposing the mask over the torso image; displaying an indicator corresponding to a location of each of the plurality of sub-regions on the torso image; as well as displaying a relative position of the ultrasound probe with respect to the indicator, the indicator corresponding to a position of each of the plurality of sub-regions of the subject's body to be imaged; The method also includes using the video stream or images obtained by accurately placing the ultrasound probe to determine areas of the lung where COVID-19 pathology or pneumonia pathology is present.

2. The method according to claim 1, further comprising: identifying a relative position of the ultrasound probe over the subject's body relative to the plurality of sub-regions to be imaged; as well as A signal is generated indicative of inaccurate positioning of the ultrasound probe over at least one of the plurality of sub-regions.

3. The method according to claim 1, further comprising: identifying a relative position of the ultrasound probe over the subject's body relative to the plurality of sub-regions to be imaged; generating different signals indicating accurate positioning of the ultrasound probe over at least one of the plurality of sub-regions of the subject's body; and Guided ultrasound imaging of a pathology of the subject is performed. The method of claim 1 , wherein the region of the subject's body to be imaged is the subject's lungs. The method according to claim 1 , wherein the subregion of the subject's body to be imaged includes a peripheral region or a lower region of the lung.

6. The method of claim 1, wherein capturing the video of the subject's body comprises using a portable handheld device having a camera and a display screen. The method of claim 6 , wherein the portable handheld device is a mobile phone.

8. The method of claim 1, wherein generating the torso image of the subject's body comprises inputting a subject video into a deep learning module trained to detect the subject's body position and generate the torso image.

9. The method of claim 1, wherein registering the video with the anatomical atlas comprises using a subset of landmarks of the subject's body detected externally by a deep learning module to register with an atlas module.

10. The method of claim 1, further comprising marking a plurality of grid lines on the mask and illustrating the sub-regions within the grid lines for imaging.

11. The method according to claim 1, wherein displaying the relative positions of the ultrasound probe with respect to the multiple sub-regions of the subject's body further comprises displaying the sub-regions to be imaged to an operator on a mobile phone display screen, and controlling the movement of the ultrasound probe by the operator.

12. The method according to claim 1 further includes employing an artificial intelligence module, wherein the artificial intelligence module is configured to detect the relative position of the ultrasound probe relative to at least one of the multiple sub-regions to be imaged, and to send a signal indicating an inaccurate positioning of the ultrasound probe over at least one of the multiple sub-regions of the subject's body.

13. The method of claim 12, wherein the artificial intelligence module is further configured to analyze the real-time stream from the ultrasound probe to detect the presence of infectious pathology in the lung.

14. A system for guided coverage and automatic pathology detection of a subject's body, the system comprising: a portable device comprising a camera and a display screen configured to capture a video stream of the subject's body; an ultrasound probe positioned over the area of the subject's body to be imaged and connected to the portable device; as well as a computer system connected to the portable device and configured to receive a plurality of ultrasound images and a video stream of the subject's body, wherein the computer system comprises: processor; a memory connected to the processor; at least one artificial intelligence module disposed on the memory and configured to generate a torso image of the subject's body; an atlas module disposed on the memory and configured to generate a mask of the region of the subject's body to be imaged and superimpose the mask over the torso image of the subject's body, the region including a plurality of sub-regions; The computer system is further configured to display an indicator corresponding to the position of each of the multiple sub-regions on the torso image of the subject's body, and to display the relative position of the ultrasound probe with respect to the indicator; the computer system is further configured to use the multiple ultrasound images or the video stream to determine a lung region where COVID-19 pathology or pneumonia pathology is present.

15. The system of claim 14, wherein the artificial intelligence module is configured to identify a relative position of the probe with respect to the sub-region to be imaged, and to generate a signal if the ultrasound probe is not accurately positioned over at least one of the plurality of sub-regions. 16 . The system of claim 15 , wherein the artificial intelligence module is configured to generate a signal to perform imaging of the subject if the ultrasound probe is accurately positioned over at least one sub-region among the plurality of sub-regions.

17. The system of claim 15, wherein the subregion comprises a peripheral region or a lower region of the lung.

18. The system of claim 15, wherein the artificial intelligence module is configured to analyze a video stream or image of the pathology obtained by the ultrasound probe to detect the sub-region where COVID-19 or pneumonia pathology is present.

19. The system of claim 14, wherein the portable device is a mobile phone.

20. The system of claim 14, wherein the atlas module is configured to mark a plurality of grid lines above the mask and indicate the sub-regions for imaging within the grid lines.

Images