Rescan assistance and path planning for blind sweep ultrasound

Abstract

Apparatuses, systems, and methods for automatic rescan assistance and blind sweep ultrasound protocol scan path planning may predict (514), using an artificial intelligence (AI) model, one or more anatomies in a first path of a sweep of a blind sweep ultrasound imaging protocol and may receive data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol. The system may determine (516), using the AI model, that the data does not include the predicted one or more anatomies. The system may predict (524), using the AI model, a second path of the sweep of the blind sweep ultrasound imaging protocol, wherein the second path of the sweep of the blind sweep ultrasound imaging protocol includes the predicted one or more anatomies, and may output the second path of the blind sweep ultrasound imaging protocol.

Description

2024PF00292RESCAN ASSISTANCE AND PATH PLANNING FOR BLIND SWEEP ULTRASOUNDTECHNICAL FIELD

[0001] The present disclosure relates to acquiring and analyzing medical imaging data. For example, embodiments herein relate to acquiring, and analyzing blind sweep ultrasound imaging data using artificial intelligence (Al) model(s).BACKGROUND

[0002] Performing various ultrasound imaging protocols is a vital component of high-quality obstetric care. In rural and under resourced communities, the scarcity of ultrasound imaging results in a considerable gap in the healthcare for fetal care before, during, and after pregnancy. A blind sweep imaging protocol, for pre-clinical assessment of the fetus, may be taught with minimal difficulty to rural health workers without prior ultrasound experience (e.g., midwives, nurses). The blind sweep protocol involves maneuvering an ultrasound probe over a predetermined grid. The user performing the blind sweep ultrasound scan protocol may not need significant knowledge of anatomy or technical skill to produce useful blind sweep ultrasound scan data. For example, a blind sweep ultrasound scan imaging obstetric protocol may be based on free-hand sweeping (transverse and longitudinal) over a grid using a portable ultrasound scanner as performed by a non-specialist. Often, the non-specialist healthcare provider has no access to the images, hence the term “blind sweep.” The ultrasound images may then be provided to a radiologist or other clinician for analysis.

[0003] However, in some cases, the blind sweep ultrasound imaging protocol may produce incomplete data as the grid that is being swept over may not be adequate with respect to the patient, or the anatomical characteristics of the patient. As a result, the blind sweep protocol may not detect / identify (i.e., misses) certain anatomies of the patient and / or the fetus. Thus, the performed blind sweep protocol may be of little or no use for analysis in determining health of the fetus or patient, nor determining other statistics of interest.

[0004] In current practice, there is no way to determine, while performing the blind sweep protocol, if the blind sweep is detecting / identifying anatomies of the fetus, of the patient carrying the fetus or missing anatomies all together. The incomplete or poor quality images acquired during the blind sweep scan may not be discovered until the images and data are reviewed later by another healthcare worker (e.g., radiologist, obstetrician). The healthcare worker would have to contact the rural health worker, and the rural health worker would have to revisit the patient to perform another blind sweep scan. This delays care and increases the time the rural health worker and patient needs to devote to scans. Further,2024PF00292 in remote areas or where rural health workers have large caseloads, making additional appointments for scans may be difficult. Accordingly, determining quality of the blind sweep scan during the initial scan is desirable.SUMMARY

[0005] Apparatuses, systems, and methods for automatic rescan assistance and blind sweep ultrasound protocol scan path planning may predict, using an artificial intelligence (Al) model, one or more anatomies in a first path of a sweep of a blind sweep ultrasound imaging protocol and may receive data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol. The system may determine, using the Al model, that the data does not include the predicted one or more anatomies. The system may predict, using the Al model, a second path of the sweep of the blind sweep ultrasound imaging protocol, wherein the second path of the sweep of the blind sweep ultrasound imaging protocol includes the predicted one or more anatomies, and may output the second path of the blind sweep ultrasound imaging protocol.

[0006] In accordance with at least one example disclosed herein, an ultrasound imaging system is disclosed. The ultrasound imaging system comprises a display and at least one processor configured to predict, using an artificial intelligence (Al) model, one or more anatomies in a first path of a sweep of a blind sweep ultrasound imaging protocol. The at least one processor is further configured to receive data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol. The at least one processor is further configured to make a determination, using the Al model, whether the data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol includes the predicted one or more anatomies. The at least one processor is further configured to predict, using the Al model, a second path of the sweep of the blind sweep ultrasound imaging protocol, based on the determination. The at least one processor is further configured to cause the second path of the blind sweep ultrasound imaging protocol to be provided on the display.

[0007] In some embodiments, the at least one processor is further configured to cause the first path to be provided on the display with the second path.

[0008] In some embodiments, the predicted second path comprises an updated starting point.

[0009] In some embodiments, the ultrasound imaging system further comprises an ultrasound probe configured to acquire the data corresponding to the first path, and the at least one processor is further configured to determine a location of the ultrasound probe based, at least in part, on the determination. In some such embodiments, the at least one processor is further configured to cause the location of the ultrasound probe to be provided on the display .

[0010] In some embodiments, the ultrasound imaging system further comprises a speaker, wherein the at least one processor is further configured to cause the speaker to output an audio indication, in2024PF00292 response to determining that the data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol does not include the predicted one or more anatomies.

[0011] In some embodiments, the at least one processor is further configured to determine the first path of the sweep of the blind sweep ultrasound imaging protocol based on one of an analysis of an image of a subject acquired by a camera or a fundal height measurement of the subject provided by a user.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] FIG. 1 illustrates a block diagram of an ultrasound imaging system arranged in accordance with principles of the present disclosure.

[0013] FIG. 2 illustrates an example image of a blind sweep protocol.

[0014] FIG. 3 illustrates an example image of a blind sweep protocol grid.

[0015] FIG. 4 illustrates an example of a path planning flow diagram for rescanning paths of a blind sweep protocol according to embodiments herein.

[0016] FIG. 5 illustrates a path planning flow diagram for rescanning paths of a blind sweep protocol when no camera is used according to embodiments herein.

[0017] FIG. 6 illustrates example image of expected anatomy in a path of a blind sweep protocol and the corresponding scan output according to embodiments herein.

[0018] FIG. 7 illustrates an example of a high level path planning flow diagram according to embodiments herein.

[0019] FIG. 8 illustrates an example of determining a blind sweep protocol grid when using a camera according to embodiments herein.

[0020] FIG. 9 illustrates an example of a grid correction algorithm according to embodiments herein.

[0021] FIG. 10 illustrates a method of performing a blind sweep protocol according to embodiments herein.DETAILED DESCRIPTION

[0022] The following description of certain examples is illustrative in nature and is in no way intended to limit the disclosed technology or its applications or uses. In the following detailed description of examples of the present apparatuses, systems, and methods, reference is made to the accompanying drawings, which form a part hereof, and in which are shown by way of illustration specific examples in which the described apparatuses, systems, and methods may be practiced. These examples are described in sufficient detail to enable those skilled in the art to practice the presently disclosed apparatuses, systems, and methods, and it is to be understood that other examples may be2024PF00292 utilized and that structural and logical changes can be made without departing from the spirit and scope of the present disclosure. Moreover, for the purpose of clarity, detailed descriptions of certain features will not be discussed when they would be apparent to those with skill in the art, so as not to obscure the description of the present technology. The following detailed description is therefore not to be taken in a limiting sense, and the scope of the present technology is defined only by the appended claims.

[0023] Users of medical imaging systems, such as ultrasound imaging systems, face technical challenges related to acquiring images using standard clinical protocols. For example, it may take two or more years to train sonographers to operate ultrasound imaging systems. The sonographer must be able to recognize anatomy in images and manipulate the probe in order to obtain images from standard planes, which vary based on the type of exam. The sonographer may also be required to take various measurements either from images (e.g., left ventricle internal diameter from a standard plane image) or other ultrasound data (e.g., blood flow rate from Doppler or m-mode data). Recently, some ultrasound systems are capable of providing support for sonographers or clinicians with less familiarity with receiving ultrasound data. For example, a user can perform guided sweeps in which the user follows guidance provided via a device or system to reach a position or orientation for capturing an image of a standard plane. As a result, the less experienced user may be able to acquire various measurements used for health monitoring or diagnosis.

[0024] In prenatal care settings, fetal growth may be detected and monitored to determine if the fetus is developing normally. Views of the fetus and its organs may be obtained via ultrasound imaging and analyzed for abnormalities. The user may acquire from the images standard measurements of a fetus such as a head circumference measurement, and the measurements may be compared to expected ranges.

[0025] In some other examples, ultrasound imaging data can be acquired (e.g., by a novice user) by moving an ultrasound probe according to a predetermined grid (termed “blind sweeps”). The blind sweep may be made up of various grid patterns such as a 3x3 pattern, a 5x5 pattern or other grid patterns of the sort (where the horizontal paths are termed “Cl”, “C2”, “C3” and so on starting from the bottom of the patient’s abdomen, “Cl”, to the top of the subject’s abdomen, “Cn”, and vertical sweep paths are termed “R” “M”, and “L” were the “R” path corresponds to the right side of the patient, the “M” path corresponds to the middle of the patient (e.g., a vertical line passing through the navel), and the “L” path corresponds to the left side of the patient). Such blind sweep ultrasound scanning protocols are useful, for example, in resource-constrained settings or in environments where experienced or trained users of ultrasound imaging systems may not be available. Additionally, blind sweep ultrasound scanning protocols may be useful for acquiring ultrasound imaging data for further processing by one or more machine learning models or other analysis algorithms.2024PF00292

[0026] However, existing procedures employing the “blind sweep” ultrasound scanning protocol to acquire ultrasound imaging data, may produce unusable data as the blind sweep may not detect / identify certain anatomies of the patient or of the fetus, due to the predetermined nature of the blind sweep grid and lack of rescan procedures in the blind sweep protocol. Currently, there are no procedures to correct the blind sweep protocol (i.e., each sweep of the blind sweep protocol) mid scan in order to produce useful blind sweep ultrasound scan data. Additionally, in current practice, there are no processes for automated grid planning based on, for example, a patient’s abdomen size. Users may manually decide to capture the scans. However, later during the usage of data corresponding to the blind sweep protocol it may be found that the scans do not have enough information for clinical assessment / analysis.

[0027] The present disclosure describes a system and related methods for automatic rescan assistance and path planning for blind sweep ultrasound scanning techniques using, for example, an artificial intelligence (Al) model (or analysis algorithms of the sort such as artificial intelligence (Al) models and machine learning models). In some embodiments, expected anatomies may be predicted, using an Al model, within a first path of a blind sweep ultrasound protocol. For example, in a Cl path it may be expected that the bladder of the patient, the cervix of the patient and the internal orifice (os) of the cervix of the patient are to be identified. Then, the path of the blind sweep protocol may be scanned over (i.e., the user may sweep an ultrasound probe along the path) to acquire ultrasound data.

[0028] The data corresponding to said path of the blind sweep protocol may be analyzed to identify, using an Al model, what anatomies have been detected, as compared to what anatomies are expected to be in the path scan.

[0029] If the expected anatomies are not found in the data of the sweep (or parts of the anatomies are found that are incomplete or are not of diagnostic value) or if the data corresponding to the scan is found to be faulty, the scan may be determined to be “inaccurate” or “incorrect” (as determined, in some cases, by the Al model) where the path may need rescanning to obtain the correct / non -faulty data. As a result, in some embodiments, the technology may lead the user through a rescanning procedure where the path used for the initial scan may need to be corrected / updated, using the Al model before being rescanned.

[0030] The rescanning procedure may begin with the Al model determining that the path of the blind sweep protocol needs rescanning. In some cases, the Al model may determine that the path of the blind sweep protocol should be adjusted. For example, a horizontal path may need to be performed higher or lower on the patient's body / abdomen as compared to the previous scan path. In another example, a vertical path may need to be performed more to the left or right of the previous scan path. In some other cases, the Al model may determine that the path was correct (i.e., trajectory and direction), however the starting point of the path scan was incorrect and should be updated. In yet some other cases, the Al model may determine that the trajectory of the path of the blind sweep protocol is incorrect and needs2024PF00292 updating. In other words, in some embodiments, the Al model may determine the scan path should be adjusted mid-sweep rather than after the sweep has been performed. In further yet some other cases, the Al model may determine a positional relationship between the predicted expected anatomies and the anatomies (or portions of anatomies) that have been detected / identified in the scanned path and compare the determined positional relationship to a probabilistic positional relationship between anatomies (that most likely is to occur). The comparison may then be used to update the path and / or blind sweep protocol grid if necessary.

[0031] In some implementations, determinations of corrections to the scan path made by the Al model may be based on the anatomies found or not found in a first scan of the path of the blind sweep ultrasound protocol, or on anatomies predicted to be expected in the path, as predicted by the Al model. In some examples, the scan path correction determination made by the Al model may be based on previous patient data, where it may be known that the patient has abnormalities in their anatomies, such as positional abnormalities inside the patient or size abnormalities of the anatomy. In some other examples, the scan path correction determinations may be based on previous historical data obtained from previous other patients, where said data may be used to train the Al model.

[0032] Accordingly, the Al model may output an updated path that has a better chance of including the predicted expected anatomies to the user performing the scanning of the path of the blind sweep ultrasound protocol. The updated path may be output to a display viewable to the user in some examples. The updated path may have a different starting point, a different trajectory, or a differing path altogether as compared to the original path used in the original scan. The system may indicate to the user to perform the scan on the updated path, where the same analysis, using the Al model, is performed on the updated path.

[0033] In some instances, the user may be provided the updated path through visual or audible indications. For example, if a camera is able to be used, the updated path of the blind sweep protocol may be provided as an overlay on a live image of the patient’s abdomen / front provided on a display to guide the user of where to perform the updated sweep. In some other examples, if no camera is being used, the updated path may be displayed on a still image of the patient or graphic representing the patient on a display (e.g., computer screen or tablet). In yet some other examples, an audio indication may be played to the user to start scanning the updated path. An audio indication may be played to indicate the user is not following the updated path, and the user should either fix the position of the probe or perform a rescan of the updated path.

[0034] In some examples, the system may indicate to the user mid scan of the path that the path does not contain the anatomies expected to be in the path (or has passed the expected position of the expected anatomies) and the system may update the path mid-scan. Such updating mid scan may2024PF00292 include, for example, indicating to the user to move the probe over a different arc, change the position of the probe when completing the path scan, and / or change the angle of the probe.

[0035] In some cases, the entire blind sweep protocol grid may be updated based on the updating / rescanning of one path of the blind sweep protocol grid. For example, if the “Cl” path needs updating / rescanning as it was determined that it was performed too low and need to be performed higher vertically on the patient’s body (e.g., farther from the pubic bone and closer to the fundus of the uterus), the entire blind sweep protocol grid may be updated to indicate other scan paths should be performed higher vertically on the patient’s body.

[0036] In some cases of failure of a scan path to obtain the expected anatomies, the system may provide an automated rescan, where the system may assist the user through capturing checkpoints to assist acquiring useful scan data from the blind sweep protocol. As a result, the user may see live updating of the path of the blind sweep protocol on the display, which may improve the user’s ability to perform the blind sweep protocol.

[0037] If the expected anatomies were found in the data corresponding to the path of the blind sweep, the scan may be considered “correct” or “accurate.” The system may provide an indication of the successful scan path (e.g., visual and / or audio signal). The user may move onto the next path of the blind sweep ultrasound protocol (e.g., if Cl was found correct, the user may move on to “C2”). The same process and / or analysis may be performed on the next path and may be repeated where expected anatomies are predicted, and the path of the blind sweep protocol ultrasound scan is performed. In some cases, if only a portion of an anatomy is found and / or not all expected anatomy is found the scan may still be considered “correct” or "accurate" as it may not be feasible to obtain a perfect scan of all anatomies due to distortion, anatomical abnormalities, exam conditions, as other reasons of the sort. As a result, the Al model may determine that the scan is useful (even with partial anatomies identified), thus prompting the user to move on and scan the subsequent path of the blind sweep protocol. In some instances, if the scan of the same path is performed multiple times and is still failing (e.g., after “N” number of attempts as configured by the system), the user may be prompted by the system to perform a scan of the second path as to obtain further data that may be useful in grid correction and for predicting the first path. In some instances, the user may be prompted by the system to return to rescan the first path that was skipped.

[0038] The scanning and if necessary, rescanning procedure, may be repeated until all paths of the blind sweep ultrasound protocol are scanned / completed and are determined to be “correct” and / or “accurate.” As noted, “correct” and / or “accurate” indicates the sweep data contains useful data for determining the health of the fetus and / or patient based on blind sweep data.

[0039] Optionally, the user may configure what is to be considered “correct” and / or “accurate” data by the technology. For example, the user may configure the data to be “correct” if it includes a certain2024PF00292 percent of the anatomies intended to be detected and identified in the scan (e.g., 50% or 70%). In some other examples, the user may configure the data to be “correct” if it includes certain “necessary” anatomies that must be detected / identified when performing the blind sweep protocol.

[0040] Alternatively or additionally, in some embodiments, instead of checking for expected anatomies in the path scan data, a quality check may be performed on the path scan data. The quality check may include checking if a resolution of the path scan data meets a certain threshold, or checking if a signal to noise ratio of the path scan data meets a certain threshold. If the quality check fails, a rescan and updating of the path being scanned may be needed. If the quality check does not fail, the scan of the path of the blind sweep protocol may be determined to be “accurate” and / or “useful” and the user may be prompted to perform a scan of the subsequent path of the blind sweep protocol grid.

[0041] Advantages of the disclosed technology may include introducing system(s) and method(s) to correct blind sweep paths as to acquire complete useful data while performing blind sweep ultrasound protocols. For example, due to the current static / predetermined nature of the blind sweep ultrasound protocol grid, blind sweep protocols may not detect / identify certain anatomies of the patient or the fetus, thus making the scan of limited clinical value. Technologies herein introduce updating of each path of the blind sweep protocol, while performing the blind sweep protocol, as to acquire complete and useful data during the exam. Technology disclosed herein may guide an inexperienced users to capture more useful data when performing blind sweep protocols as the system or method may output visual and / or audible indications for the user to follow to acquire more accurate blind sweep scans. This may result in more clinically useful data that can be analyzed by other healthcare workers and reduce the need for follow-up exams to rescan patients.

[0042] FIG. 1 illustrates a block diagram of an ultrasound imaging system 100 arranged in accordance with principles of the present disclosure. In the ultrasound imaging system 100 of FIG. 1, an ultrasound probe 112 includes a transducer array 114 for transmitting ultrasonic waves and receiving echo information. The transducer array 114 can be implemented as a linear array, convex array, a phased array, and / or a combination thereof. The transducer array 114, for example, can include a two- dimensional array (as shown) of transducer elements capable of scanning in both elevation and azimuth dimensions for 2D and / or 3D imaging. The transducer array 114 can be coupled to a microbeamformer 116 in the probe 112, which controls transmission and reception of signals by the transducer elements in the array. In this example, the microbeamformer 116 is coupled by the probe cable to a transmit / receive (T / R) switch 118, which switches between transmission and reception and protects the main beamformer 122 from high-energy transmit signals. In some embodiments, the T / R switch 118 and other elements in the system can be included in the ultrasound probe 112 rather than in a separate ultrasound system base. In some embodiments, the ultrasound probe 112 may be coupled to the ultrasound imaging system via a wireless connection (e.g., WiFi, Bluetooth).2024PF00292

[0043] The transmission of ultrasonic beams from the transducer array 114 under control of the microbeamformer 116 is directed by the transmit controller 120 coupled to the T / R switch 118 and the beamformer 122, which receives input from the user’s operation of the user interface (e.g., control panel, touch screen, console) 125. The user interface 125 may include soft and / or hard controls. One of the functions controlled by the transmit controller 120 is the direction in which beams are steered. Beams may be steered straight ahead from (orthogonal to) the transducer array 114, or at different angles for a wider field of view. The partially beamformed signals produced by the microbeamformer 116 are coupled via channels 115 to a main beamformer 122 where partially beamformed signals from individual patches of transducer elements are combined into a fully beamformed signal. In some embodiments, microbeamformer 116 is omitted and the transducer array 114 is coupled via channels 115 to the beamformer 122. In some embodiments, the system 100 can be configured (e.g., include a sufficient number of channels 115 and have a transmi t / receive controller programmed to drive the transducer array 114) to acquire ultrasound data responsive to a plane wave or diverging beams of ultrasound transmitted toward the subject. In some embodiments, the number of channels 115 from the ultrasound probe may be less than the number of transducer elements of the transducer array 114 and the system can be operable to acquire ultrasound data packaged into a smaller number of channels than the number of transducer elements.

[0044] The beamformed signals are coupled to a signal processor 126. The signal processor 126 can process the received echo signals in various ways, such as bandpass filtering, decimation, I and Q component separation, and / or harmonic signal separation. The signal processor 126 can also perform additional signal enhancement such as speckle reduction, signal compounding, and noise elimination. The processed signals are coupled to a B-mode processor 128, which can employ amplitude detection for the imaging of structures in the body. The signals produced by the B-mode processor 128 are coupled to a scan converter 130 and a multiplanar reformatter 132. The scan converter 130 arranges the echo signals in the spatial relationship from which they were received in a desired image format. For instance, the scan converter 130 can arrange the echo signal into a two-dimensional (2D) sector-shaped format, or a pyramidal three-dimensional (3D) image. The multiplanar reformatter 132 can convert echoes, which are received from points in a common plane in a volumetric region of the body into an ultrasonic image of that plane, as described in U.S. Pat. No. 6,443,896 (Detmer).

[0045] A volume Tenderer 134 converts the echo signals of a 3D data set into a projected 3D image as viewed from a given reference point, e.g., as described in U.S. Pat. No. 6,530,885 (Entrekin et al.). In some implementations, the system 100 can additionally or alternatively be configured to perform 3D and / or 4D acquisitions, such as protocolized 3D / 4D acquisitions. The 2D or 3D images can be coupled from the scan converter 130, multiplanar reformatter 132, and volume Tenderer 134 to at least one processor 137 for further image processing operations. For example, the at least one processor 137 can2024PF00292 include an image processor 136 configured to perform further enhancement and / or buffering and temporary storage of imaging data for display on an image display 138. However, in some embodiments, the display 138 may not display ultrasound images to the user.

[0046] The display 138 can include a display device implemented using a variety of display technologies, such as LCD, LED, OLED, or plasma display technology. The at least one processor 137 can include a graphics processor 140, which can generate graphic overlays for display with the ultrasound images. These graphic overlays can contain, e.g., standard identifying information such as patient name, date and time of the image, imaging parameters, and the like. For these purposes the graphics processor 140 receives input from the user interface 125, such as a typed patient name. The user interface 125 can also be coupled to the multiplanar reformatter 132 for selection and control of a display of multiple multiplanar reformatted (MPR) images.

[0047] The user interface 125 can include one or more mechanical controls, such as buttons, dials, a trackball, a physical keyboard, and others, which may also be referred to herein as hard controls. Alternatively or additionally, the user interface 125 can include one or more soft controls, such as buttons, menus, soft keyboard, and other user interface control elements implemented, for example, using touch-sensitive technology (e.g., resistive, capacitive, or optical touch screens). One or more of the user controls can be co-located on a control panel 124. For example, one or more of the mechanical controls can be provided on a console and / or one or more soft controls can be co-located on a touch screen, which can be attached to or integral with the console. The display 138 and the user interface 125 can be included in an I / O component, via which outputs are provided by the system 100 and / or inputs are received by the system 100.

[0048] In some implementations, the user interface 125 can receive inputs and provide outputs of the disclosed system. For example, the user interface 125 can receive a user input specifying an exam type.

[0049] The at least one processor 137 (e.g., the image processor 136, the graphics processor 140, or a different processor) can perform functions associated with acquiring and configuring medical imaging data, as described herein. For example, the at least one processor may implement one or more Al models or other machine learning or neural networks for updating paths of a blind sweep protocol.

[0050] Although described as separate processors, it will be understood that the functionality of any of the processors described herein can be implemented in a single processor (e.g., a CPU or GPU implementing the functionality of processor 137) or fewer number of processors than described in this example. In some embodiments, the at least one processor 137 can be hardware -based (e.g., include multiple layers of interconnected nodes implemented in hardware). In some embodiments, the at least one processor 137 can be implemented at other processing stages, e.g., prior to the processing performed by the image processor 136, volume Tenderer 134, multiplanar reformatter 132, and / or scan converter 130. In some embodiments, the at least one processor 137 can be implemented to process2024PF00292 ultrasound data in the channel domain, beamspace domain (e.g., before or after beamformer 122), the IQ domain (e.g., before, after, or in conjunction with signal processor 126), and / or the k-space domain. As described, in some embodiments, functionality of two or more of the processing components (e.g., beamformer 122, signal processor 126, B-mode processor 128, scan converter 130, multiplanar reformatter 132, volume Tenderer 134, at least one processor 137, image processor 136, graphics processor 140, etc.) can be combined into a single processing unit and / or divided between multiple processing units. The processing units can be implemented in software, hardware, or a combination thereof. For example, the at least one processor 137 can include one or more graphical processing units (GPU). In another example, beamformer 122 can include an application specific integrated circuit (ASIC).

[0051] The at least one processor 137 can be coupled to one or more computer-readable media (e.g., memory 142) included in the system 100, which can be non-transitory. The one or more computer- readable media can carry instructions and / or a computer program that, when executed, cause the at least one processor 137 to perform operations described herein. A computer program can be stored / distributed on any suitable medium, such as an optical storage medium or a solid- state medium supplied together with or as part of other hardware, and can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Furthermore, embodiments can take the form of a computer program product accessible from a computer- readable medium providing program code for use by or in connection with a computer or any device or system that executes instructions. For the purposes of this disclosure, a computer-readable medium can generally be any tangible apparatus that can contain, store, communicate, propagate, and / or transport the program for use by or in connection with the instruction execution device. The computer- readable medium can be, for example, without limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, and / or a propagation medium. Non-limiting examples of a computer readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk, and / or an optical disk. Optical disks can include compact disk read only memory (CD-ROM), compact disk-read / write (CD-R / W), and / or DVD.

[0052] In some embodiments, while shown separately, many of the components of system 100 may be included within the probe 112. In some embodiments, some of the components of system 100 may be included in a tablet or other portable computing device. In some applications, this may provide an ultrasound imaging system that may be more portable and / or more suitable for care in remote / rural environments compared to traditional cart -based ultrasound systems. An example of such a portable system is the Philips® Lumify® system. However, this is merely an example, and the disclosure is not limited to this specific system.2024PF00292

[0053] The system 100 may be capable of performing the techniques disclosed herein according to principles of the present disclosure. However, in some instances, the system 100 may acquire the blind sweep data, and that data may be provided to a computing system with one or more processors (e.g., signal processor, image processor, graphics processor, multiplanar reformatter, volume Tenderer), a non-transitory computer readable medium, and a user interface. The computing system may perform some or all of the remaining non-image acquisition functions in these embodiments. Accordingly, while the examples provided herein describe data analysis functions being performed by an ultrasound imaging system, such as system 100, the disclosure is not limited to these embodiments.

[0054] FIG. 2 illustrates an example of a blind sweep protocol 200 according to at least one embodiment of the present disclosure. In the example shown in FIG. 2, an audio guidance 214 may be used guide the user 216 through performing the blind sweep protocol 206 on a patient 218 with an ultrasound probe (e.g., probe 112). The audio guidance 214 may be stored in a memory, such as memory 142 and played on a speaker, which may be part of a user interface, such as user interface 125. The blind sweep protocol 206 may be stored in the memory in some examples. Based on the data collected from performance of the blind sweep protocol 206, various medical images may be generated 212 (i.e., data corresponding to the blind sweep protocol 206). The images may be triaged and / or monitored 208. The images may also be stored in the memory in some examples. In some examples, the images may be provided on a display to the user 216, such as display 138. However, in other examples, the user 216 may not have access to the images acquired from the blind sweep protocol. The triaged and / or monitored 208 data may be indicated to the user 216 in the form of a visual representation 210 in a tablet and / or display 202. The visual representation 210 may include a representation of the patient’s 218 abdomen, a representation of the paths to be followed for the blind sweep protocol 206, an indication as to whether a path has been successfully acquired, an indication of a corrected path, or a combination thereof. This data may be received by and displayed on a tablet 202 and / or other display that the user 216 may utilize while performing the blind sweep protocol 206.

[0055] In some cases, path planning 204 may be performed (based on previous blind sweep protocol scans or failed blind sweep protocol scans) for current or subsequent blind sweep protocol 206 scans. According to embodiments disclosed herein, one or more Al models may analyze data (e.g., images) acquired during the blind sweep protocol 206 to determine if required anatomies were detected. If required anatomies were not detected, the Al model may perform path planning (e.g., determine a corrected path to sweep) for the blind sweep protocol. The corrected path from the path planning 204 may be provided via the visual representation 210 and / or audio guidance 214. In some embodiments, the analysis and corrected path (if needed) may be performed during a scanning of a path. In some embodiments, the analysis and corrected path (if needed) may be performed after scanning of a path.2024PF00292However, the path planning 204 may be performed once the previous blind sweep protocol is completely scanned (i.e., each path of the blind sweep protocol is scanned).

[0056] FIG. 3 illustrates an example of a blind sweep protocol grid 300 according to at least one embodiment of the present disclosure. A blind sweep protocol grid 300 represents the paths a user should sweep the probe over a subject while performing the blind sweep protocol to obtain ultrasound imaging data. In some examples, the protocol grid 300 may be provided to a user, such as user 216 on a display such as display 138. In the example shown in FIG. 3, the protocol grid 300 illustrates paths to be traced over a subject’s abdomen represented by the oval 314. The example shown in FIG. 3 is a 3x3 grid, but other examples may include a 4x4 grid, a 3x5 grid, a 5x5 grid, a 6x6 grid and so on. The number of horizontal and / or vertical paths for the blind sweep protocol grid 300 may be based on the fundal height of the patient, as measured by the user or determined by a technology such as a camera. The example blind sweep protocol grid 300 illustrated in FIG. 3 is a 3x3 blind sweep protocol grid including three horizontal paths and three vertical paths. In some blind sweep protocols, the paths may be acquired in a particular order. For example, the blind sweep protocol using the blind sweep protocol grid 300 may begin with a user scanning the Cl 302 path, then scanning the C2 304 path and then scanning the C3 306 path. Subsequently, the user may scan the R 308 path, then the M 310 path, and then the L 312 path.

[0057] FIG. 4 illustrates an example of a path planning flow diagram 400 for rescanning paths of a blind sweep protocol according to embodiments herein.

[0058] The path planning flow diagram 400 may be performed using the system 100 of FIG. 1.

[0059] In some embodiments, for blind sweep protocol path planning and / or rescanning, a user may take an image or video of patient’s abdomen and / or enter the gestational age (GA) and / or fundal height. The image(s), GA, and / or fundal height may be provided as an input to get an initial blind sweep protocol grid (e.g., protocol grid 300), which may be a factory template. The grid may be displayed as an overlay the image of the patient or on a graphic representing the patient. In some cases, Al based anatomy and landmark detection may be used during scanning. Based on the anatomy and landmark detection, a user may be guided to move the ultrasound probe if necessary anatomy is not detected (e.g., due to location of fetus, patient’s anatomy outside expected range, shadows, distortion, inadequate contact of the probe, or other reasons of the sort). Automatic customized path (e.g., grid) correction may be determined by the Al model and displayed on a display to guide the user to rescan one or more paths of the blind sweep protocol.

[0060] The path planning flow diagram 400 may begin by receiving an image and / or various statistics of the patient (e.g., gestational age, weight, etc.), at block 402, for use by a scanner, at block 404. The scanner may be included in a processor, such as an image processor or other processor, which may analyze an image of the patient and / or statistics of the patient to determine a suitable blind sweep2024PF00292 protocol grid. An image of the blind sweep protocol grid may be provided on a display (e.g., display 138) to guide the user performing the blind sweep protocol at block 406.

[0061] In some embodiments, data corresponding to a previous failed path scan of the blind sweep protocol, at block 408, in combination with data corresponding to follow-up path scans, at block 410, may be used as input(s) to an Al model. The Al model may be used for anatomical landmark detection for a path (i.e., predicting anatomies expected to be in a path and detecting if anatomies within the path of the blind sweep protocol) at block 412. Landmark registration may be performed at block 414, using the predicted expected anatomies in the scan and detected anatomies in the scan received from the Al model at block 412. If predicted anatomy in a scan path are not detected, planning of a correction (e.g. determining / updating) of the path of the blind sweep protocol may be performed at block 416 based on the landmark registration at block 414 and the expected predicted anatomies, as predicted by the Al model at block 412. The user may receive a guidance indication at block 418 from the system indicating the updated path of the blind sweep protocol. Such guidance indication at block 418 may be a visual indication, an audio indication, or a gesture indication received as vibrations in the probe communicating with the ultrasound system as to assist the user in performing the scan of the path of the blind sweep protocol. In some examples, an updated path 422 (as illustrated by the solid arrow line) with an updated starting position 426 may be displayed at block 420. In some examples, the updated path 424 may be displayed conjunction with the previously performed path 424 (as illustrated by the dashed arrow) with the previous starting position 428, as to further assist the user in performing the corrected path scan.

[0062] If at blocks 412 and 414, it is determined that all necessary anatomy was acquired in the initial sweep of the path, the system may provide an indication of same on a display and / or as an audio indication. The user may then move on to performing the next path on the blind sweep protocol. In some embodiments, the system may prompt the user to begin the next path. The protocol described above may be repeated for each path of the blind sweep protocol until all paths are successfully acquired.

[0063] FIG. 5 illustrates a path planning flow diagram 500 for rescanning paths of a blind sweep protocol when no camera is used according to embodiments herein.

[0064] The path planning flow diagram 500 may be performed using the system 100 of FIG. 1.

[0065] At block 502, patient statistics (e.g., gestational age, fundal height, a picture of the patient's stomach) may be provided. The inputs may be used to plan and determine an initial blind sweep protocol grid for the patient at block 504. The blind sweep protocol grid (e.g., protocol grid 300) may be displayed to a user at block 506. In some examples, the protocol grid may be displayed on a tablet and / or other screen 508. The user may use an ultrasound probe (e.g., probe 112) to scan a first path of the blind sweep protocol based on the displayed blind sweep protocol grid at block 510. Landmark2024PF00292 registration may be performed with the current patient at block 512. In some embodiments, an Al model may be used to predict the expected anatomies to appear in the path of the blind sweep protocol being scanned. The Al model may perform landmark detection, at block 514. The Al model may detect anatomy or anatomical features in the ultrasound data of the scan path to determine a current location of the ultrasound probe. A graphic representing the probe may be shown on the tablet and / or other screen 508. This may guide the user to move the ultrasound probe along the path indicated by the protocol grid.

[0066] Alternatively, instead of identifying anatomy in the ultrasound data, the Al model may analyze data from a camera instead. For example, a camera may acquire images of the patient’s abdomen during the scan. In these embodiments, at block 514, the Al model may analyze the images from the camera to determine a current location of the ultrasound probe. A graphic representing the probe may be shown on the tablet and / or other screen 508.

[0067] At block 516, the Al model may analyze the ultrasound data from the scan path to identify what (if any) anatomies were found in the path of the blind sweep protocol. This may be done in real time, or after the scan is of the path is completed. If the expected anatomies (or anatomies of interest) were detected in the path scan (as detected by the Al model), the path scan may be determined to be successful at block 518. Although not shown in FIG. 5, when the path scan is successful, if additional paths are needed for the blind sweep protocol, the flow may return to start the scan for the next path at block 510.

[0068] Returning to block 516, if the expected anatomies were not detected no anatomies were detected at all, or only anatomies not of interest were detected, a rescan of the path may be needed as indicated by block 520. At block 522, an audio indication may be made to the user indicating that a rescan is needed. This may be done in real time (e.g., while the user is still scanning the current path) or after the scan of the path is completed. The grid correction may be performed at block 524. The Al model may update the path based on the predicted expected anatomies and the data corresponding to the previously performed path scan as part of the grid correction. In some instances, the grid correction may be performed for only one path of the blind sweep protocol grid or for all paths of the blind sweep protocol grid. For example, if the previous scan was performed too low (e.g., too close to the pubic bone), the entire blind sweep protocol grid may be adjusted to be performed higher on the patient (e.g., closer to the fundus). The updated grid may be provided to the user at block 526, for example on the tablet and / or screen 508. The flow may return to block 510 when the user starts subsequent scanning of the path of the blind sweep protocol. At block 516, optionally, a quality check may be performed on the path scan data. The quality check may include checking if a resolution of the path scan data meets a certain threshold, or checking if a signal to noise ratio of the path scan data meets a certain threshold. If the quality check fails, a rescan and updating of the path being scanned may be needed. If the quality2024PF00292 check does not fail, the scan of the path of the blind sweep protocol may be determined to be “accurate” and / or “useful” and the user may be prompted to perform a scan of the subsequent path of the blind sweep protocol grid

[0069] It should be understood that the path planning flow diagram 500 may be repeated until the scan of the path is determined to be successful, at block 518. In some cases, when such determination is made, the user may be prompted to move on and perform a scan of a subsequent path in the blind sweep protocol grid, until all paths are scanned and determined to be successful.

[0070] FIG. 6 illustrates example expected anatomy in a path of a blind sweep protocol 600 and the corresponding grid correction according to embodiments herein. The detection of the anatomy may be performed by an Al model, which may be implemented by a processor, such as image processor 136 in some embodiments.

[0071] In some embodiments, an Al model may predict expected anatomy in a path of a blind sweep protocol. For example, the Al model may predict that in the Cl 602 path scan, a cervix 610 and an internal os of the cervix 608 are expected to be detected / identified. However, if the performed first path scan 604 does not contain the cervix 610 and / or the internal os 608, the Al model may determine the sweep of the path should be repeated on a modified path. If the Al model determines the path scan contains the maternal urinary bladder 606, the Al model may determine that the original scan path in the first path scan 604 (as illustrated by the solid arrow) was too high.

[0072] Based on the detected maternal urinary bladder 606 and the predicted cervix 610 and internal os 608, the Al model may determine that the path may need to be updated as to be scanned lower as, usually the cervix 610 and the internal os 608 reside in a lower position as compared maternal urinary bladder 606. The updated path 612 (i.e., lower than the first path scan 604, as illustrated by the dashed arrow) may be calculated and provided to the user as guidance for rescanning the “Cl” path. The Al model may analyze a rescan of the path and may detect / identify that the cervix 610 and internal OS 608 are within the updated path 612 scan, and thus the scan is “accurate” and / or “correct.” The user may move on and perform the scan of the next path (i.e., the “C2” path).

[0073] In some implementations, the system may display what anatomy is being detected by the Al model in each path scan. In some implementations, the system may display ultrasound images from the path scan. For example, the ultrasound image 614 may corresponds to the “Cl” updated path 612 scan and may be provided on a display of a tablet or other screen available to the user. However, while images 614, 616, and 618 are shown as example images from a scan of a path from a blind sweep, these are presented for illustrating the analysis of the Al model and / or operations system components. The images 614, 616, and 618 may not be presented to a user on a display of the system during performance of the blind sweep protocol.2024PF00292

[0074] The rescanning procedure described for path “Cl” may be repeated for each path scan of the blind sweep protocol where different anatomies may be detected / identified based on the current path scan. For example, for the “C2” path a heart may be expected to be detected, for “C3” path a top portion of the uterine wall may be expected to be detected, for “M” path a maternal urinary bladder may be expected to be detected, for “L” path a left and top portion of the uterine wall may be expected to be detected, or for “R” path a right and top portion of the uterine wall may be expected to be detected. When the expected anatomies (or portions of the expected anatomies) are not found, an updated / correction of the path may be performed (e.g., shifting the path up or down or shifting to the left or right).

[0075] FIG. 7 illustrates an example of a path planning flow diagram 700 according to embodiments herein.

[0076] The path planning flow diagram 700 may be performed using the system 100 of FIG. 1.

[0077] In some embodiments, updating a path scan may or may not use a camera to develop a blind sweep protocol grid. For example, the path planning flow diagram 700 may receive patient statistics (e.g., GA, fundal height) at block 702. The patient statistics may be used in developing the initial blind sweep protocol grid. If the blind sweep protocol grid is being developed without a camera 704, the user may measure a fundal height of the patient manually at block 706. The user may use a tape measure. At block 708, the user or the system may select a predefined template for the blind sweep protocol grid (e.g., 3x3, 5x5) based on the measured fundal height. At block 710 the determined blind sweep protocol grid may be displayed to the user on a tablet and / or computer screen 712. If the blind sweep protocol grid is being developed with a camera 704, the blind sweep protocol grid may be determined based on data received from the camera. At block 716, the grid may be overlayed on a static or live image of the patient as received by the camera (as illustrated in FIG. 8). The grid and patient image may be displayed on the tablet and / or computer screen 712.

[0078] At block 718, the user may start the scan based on the blind sweep protocol grid (e.g., the user may begin acquiring “Cl” path). An Al model may perform landmark detection and registration at block 720. The Al model may predict the expected anatomies in a path of the blind sweep protocol. Optionally, if a camera is being used when performing the blind sweep protocol, at block 722 a camera may be started to track a probe being used in the blind sweep protocol.

[0079] During or after the path scan, the Al model may detect / identify which anatomies are in the scan of the path at block 724. If the expected anatomy (or other useful anatomy) is detected, the scan may be determined, to be successful , at block 726. The user may be prompted to perform the next path scan (e.g., “C2” if the previous path was “Cl”). If the expected anatomy (or other useful anatomy) is not detected at block 724, a rescan may be needed at block 728. The system may indicate to the user, using an audio guidance that the rescan may be needed at block 730. The system may perform grid2024PF00292 correction at block 732. In some embodiments, the Al model may determine an updated / corrected grid based, at least in part, on the detected anatomies in the path scan. The corrected grid may be provided to the user on the tablet and / or other screen 712 at block 734. In some embodiments, the corrected grid may be shown concurrently with the previous grid. The user may begin the rescan at block 718. The protocol shown in FIG. 7 may be performed until all paths of the blind sweep protocol have been obtained successfully.

[0080] In some implementations, such determination of the scan being successful or not may be done in real time. For example, the Al model may determine the ultrasound probe has passed the predicted position of the expected anatomy, the system may output an audio indication that the scan does not need to be completed and a rescan is needed based on an updated path.

[0081] FIG. 8 illustrates an example 800 of determining a blind sweep protocol grid when using a camera according to embodiments herein.

[0082] In some embodiments, when using a camera, a blind sweep protocol grid (used for path scanning) may be determined based on images of the patient. In some cases, when in an online and manual mode, a live image 802 of the patient may be taken and used to determine the blind sweep protocol grid 808. The determined blind sweep protocol grid 808 may be overlaid over the live image 802 of the patient when performing the blind sweep protocol scan. Such live image 802 may encompass a live video of the patient.

[0083] In some other cases, an image of the patient may be taken when in an offline 804 mode (i.e., not a live image). Such image may be used to determine the blind sweep protocol grid. The determined blind sweep protocol grid 808„ may be overlaid 806 on the non-live image taken while offline 804 and displayed to the user for assisting the user in performing each path scan of the blind sweep protocol. In some examples, such offline 804 image may be static and not live.

[0084] FIG. 9 illustrates an example of a grid correction algorithm 900 according to embodiments herein.

[0085] The grid correction algorithm 900 may be performed using the system 100 of FIG. 1. In some embodiments, the grid correction algorithm 900 may be performed by one or more processors of system 100, such as image processor 136 and / or graphics processor 140.

[0086] In some embodiments, the Al model may include or interact with a grid correction algorithm 900. At block 902, a user may scan with an ultrasound probe a next frame following a path guidance previously provided (either based on a path of the original blind sweep protocol grid or an updated path). At block 904, a current frame is acquired. An anatomical landmark detection (based on an edge / boundary) may be performed on the current frame (Fn) at block 906. For example, an anatomy may be detected / identified based on the edge / boundary.2024PF00292

[0087] If at block 908, an anatomy is not detected (no), the algorithm may determine whether the scan of the path is past the expected predicted anatomy position at block 910. If yes, the current path scan may be aborted (or the path scan may be rejected as being “inaccurate”) and a rescan may be performed at block 912. If the current path scan is not past the position of the expected predicted anatomy, the algorithm may return to block 902 where the user may scan the next frame.

[0088] If the anatomy is detected at block 908 (yes), the algorithm may perform a shape registration of the anatomy at block 916. The shape registration may utilize a 3D template and an anatomy reference provided at block 918 to identify the detected anatomy. The shape registration may be provided to a multi anatomy distance regression model at block 920. The regression model may identify irregular shaped anatomies or irregular shapes detected by the Al model. In some embodiments, the multi anatomy distance regression model may analyze past frames, illustrated in block 922 and past failed path scans illustrated in block 924 (i.e., past failed sweeps). The output of the regression model may be provided to a multi-anatomical distance forecasting model at block 926. The forecasting model may identify anatomies that are, for example, expected to be detected within the scan of the path or to further predict the distance between the detected / identified anatomy and other anatomies of interest not yet detected / identified. The regression model output and the forecasting model output, may be used in combination to determine directions for the next frame to be scanned in the scanning of the path of the blind sweep protocol at block 928. Such updates may either be indicated to the user for real time updating, or stored by the system to be used for updating the path when the scan is fully completed.

[0089] The algorithm may check if all anatomies have been detected at block 930. If yes, all anatomies have been detected, the sweep is determined to be successful and the user may be prompted to perform a scan of the next path at block 932. If no, all anatomies have not been detected, the algorithm may update the path and notify the user on the change / update of the path of the blind sweep protocol at block 934.

[0090] FIG. 10 illustrates a method 1000 of performing a blind sweep protocol according to embodiments herein. The illustrated method 1000 includes predicting 1002, using an Al model, one or more anatomies in a first path of a sweep of a blind sweep ultrasound imaging protocol. For example, in block 412, block 414, block, 514, block 720, and / or block 906, an Al model may predict one more anatomies.

[0091] The method 1000 further includes receiving 1004 data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol. For example, in block 408, block 510, block 718, and / or block 922 data corresponding to the first path of the blind sweep imaging ultrasound imaging protocol may be predicted

[0092] The method 1000 further includes making a determination 1006, using the Al model, whether the data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol2024PF00292 includes the predicted one or more anatomies. For example, in block 416, block 516, block 724, and / or block 930 the Al model may determine if the predicted one or more anatomies are included or not included in the data corresponding to the first path of the blind sweep ultrasound imaging.

[0093] The method 1000 further includes predicting 1008, using the Al model, a second path of the sweep of the blind sweep ultrasound imaging protocol, based on the determination. For example, in block 402, block 524, block 732, and / or block 934 the Al model may perform grid correction where a second path is predicted / determined based on the first path and the predicted anatomies that are expected to be in the path.

[0094] The method 1000 further includes outputting 1010 the second path of the blind sweep ultrasound imaging protocol to a display. For example, in block 420, block 526, block 734, and / or block 934 the updated path and updated blind sweep protocol grid may be displayed to the user.

[0095] In some embodiments of the method 1000, the blind sweep protocol comprises a plurality of paths including the first and second path, and the method 1000 further comprises shifting remaining ones of the plurality of paths based on the determining, and outputting the second path and the shifted remaining ones of the plurality of paths to the display. For example, in block 420, block 526, block 732, and / or block 934 the paths may be updated and outputted.

[0096] In some embodiments, the method 1000 further comprises receiving an image from a camera and determining, using the Al model, a location of an ultrasound probe acquiring the first sweep based on the image. Some such embodiments further comprise outputting a graphic representing the ultrasound probe on the display. For example, in block 404, and / or block 716 a camera may determine a location of an ultrasound probe.

[0097] In some embodiments, the method 1000 further comprises determining, using the Al model, a location of an ultrasound probe acquiring the first sweep based on the data corresponding to the first path of the sweep of the blind sweep ultrasound. Some such embodiments further comprise determining, using the Al model, whether the ultrasound probe is past an expected location of one of the predicted anatomies. Some further such embodiments further comprise outputting an audio indication, a visual indication, or a combination thereof, that the expected location has been passed. For example, in block 412, block 514, block 720, and / or block 910 the Al model may determine whether the ultrasound probe is past an expected location of the predicted anatomies. If the ultrasound probe is past the expected location of the predicted anatomies, the ultrasound imaging system may output an indication.

[0098] In some embodiments of the method 1000, the blind sweep protocol comprises a plurality of paths, wherein the plurality of paths is determined based, at least in part, on a fundal height provided as an input by a user. For example, in block 502 and / or block 706 the plurality of paths may be determined2024PF00292 based on a fundal height of the patient provided to the ultrasound imaging system as an input. In some cases, the fundal height of the patient may be measured by the user.

[0099] In some embodiments, the method 1000 further comprises outputting the first path with the second path. For example, in block 420, block 526, block 734, and / or block 934 the first path may be outputted with the predicted / updated second path.

[0100] In some embodiments of the method 1000, the predicted second path comprises an updated starting point. For example, in block 420, block 526, block 734, and / or block 934, the predicted second path may include an updated starting point compared to the starting point of the first path.

[0101] The Al model(s) disclosed herein may be trained using training datasets comprising ultrasound data acquired by blind sweeps and / or other techniques. The training datasets may further include labels, annotations, or other data. For example, the training dataset for the Al model may include images with anatomy of interest labelled (e.g., maternal bladder, fetal heart, fetal skull, maternal cervix) and bounding boxes around said anatomy of interest as well as an annotation of what sweep of a blind protocol scan the images were acquired from (e.g., “Cl,” “L”). In some embodiments, the data set may include full sweeps from a blind protocol data and annotated with whether the sweep is “successful / correcf ’ or “incomplete / rescan needed.” The incomplete / rescan data may be annotated with the corrective action needed (e.g., move up / down or left / right). The training data set may further be annotated with locations of the ultrasound probe for embodiments where anatomical detection is used to track a location of the ultrasound probe. In some embodiments, the data set may further include images acquired with a camera with the ultrasound probe labeled. Once the Al model has been trained, the Al model receives data comprising blind sweep data, and Al model may detect anatomies to determine whether a sweep is successful, and if necessary, determine corrections to the sweep path. In some embodiments, the Al model may output a current location of the ultrasound probe.

[0102] The Al model may be a neural network in some embodiments. The neural network may include one or more input nodes that receive training datasets. The input nodes can correspond to functions that receive the input and produce results. These results can be provided to one or more levels of intermediate nodes that each produce further results based on a combination of lower-level node results. A weighting factor can be applied to the output of each node before the result is passed to the next layer node. At a final layer, (“the output layer,”) one or more nodes can produce a value classifying the input that, once the model is trained, can be used to evaluate new data (e.g., detected anatomies). In some implementations, such neural networks, known as deep neural networks, can have multiple layers of intermediate nodes with different configurations, can be a combination of networks or machine learning / AI models that receive different parts of the input and / or input from other parts of the deep neural network, or are convolutions — partially using output from previous iterations of applying the neural network as further input to produce results for the current input.2024PF00292

[0103] The Al model can be trained with supervised or unsupervised learning. Testing data can then be provided to the Al model to assess accuracy. Testing data can be, for example, a portion of the entire dataset (e.g., 10%) held back to use for evaluation of the model. Output from the Al model can be compared to the desired or expected output for the training data and, based on the comparison, the Al model can be modified, such as by changing weights between nodes of the neural network and / or parameters of the functions used at each node in the neural network (e.g., applying a loss function). Based on the results of the Al model evaluation, and after applying the described modifications, the Al model can then be retrained to evaluate new data.

[0104] The apparatuses, systems, and methods disclosed herein may determine whether expected anatomies have been acquired by a sweep of a path in a blind sweep protocol exam. If expected anatomies are not detected, the apparatuses, systems, and methods may provide guidance to users to reacquire paths of a blind sweep protocol during the exam. The apparatuses, systems, and methods disclosed herein may determine corrected paths and provide guidance to users for sweeping along the corrected paths. The apparatuses, systems, and methods may allow more clinically useful data to be acquired by blind sweep protocols. This may reduce the need for repeat exams, improve diagnosis, and / or improve patient care.

[0105] Further, while the examples described herein relate to obstetric care, it is understood that the principles of the present disclosure may be extended to other applications where a set blind sweep protocol is used where there is anatomy predicted to be located along particular paths of the blind sweep. For example, echocardiograms where particular standard planes, chambers, and / or valves are expected to be captured by certain paths of a blind sweep protocol.

[0106] In various examples where components, systems and / or methods are implemented using a programmable device, such as a computer-based system or programmable logic, it should be appreciated that the above-described systems and methods can be implemented using any of various known or later developed programming languages, such as “Python”, “C”, “C++”, “FORTRAN”, “Pascal”, “VHDL” and the like. Accordingly, various storage media, such as magnetic computer disks, optical disks, electronic memories and the like, can be prepared that can contain information that can direct a device, such as a computer, to implement the above-described systems and / or methods. Once an appropriate device has access to the information and programs contained on the storage media, the storage media can provide the information and programs to the device, thus enabling the device to perform functions of the systems and / or methods described herein. For example, if a computer disk containing appropriate materials, such as a source file, an object file, an executable file or the like, were provided to a computer, the computer could receive the information, appropriately configure itself and perform the functions of the various systems and methods outlined in the diagrams and flowcharts above to implement the various functions. That is, the computer could receive various portions of 12024PF00292 information from the disk relating to different elements of the above-described systems and / or methods, implement the individual systems and / or methods and coordinate the functions of the individual systems and / or methods described above.

[0107] In view of this disclosure it is noted that the various methods and devices described herein can be implemented in hardware, software, and / or firmware. Further, the various methods and parameters are included by way of example only and not in any limiting sense. In view of this disclosure, those of ordinary skill in the art can implement the present teachings in determining their own techniques and needed equipment to affect these techniques, while remaining within the scope of the invention. The functionality of one or more of the processors described herein may be incorporated into a fewer number or a single processing unit (e.g., a CPU) and may be implemented using application specific integrated circuits (ASICs) or general -purpose processing circuits which are programmed responsive to executable instructions to perform the functions described herein.

[0108] Although the present system may have been described with particular reference to an ultrasound imaging system, it is also envisioned that the present system can be extended to other medical imaging systems where one or more images are obtained in a systematic manner. Accordingly, the present system may be used to obtain and / or record image information related to, but not limited to renal, testicular, breast, ovarian, uterine, thyroid, hepatic, lung, musculoskeletal, splenic, cardiac, arterial and vascular systems, as well as other imaging applications related to ultrasound-guided interventions. Further, the present system may also include one or more programs which may be used with conventional imaging systems so that they may provide features and advantages of the present system. Certain additional advantages and features of this disclosure may be apparent to those skilled in the art upon studying the disclosure, or may be experienced by persons employing the novel system and method of the present disclosure. Another advantage of the present systems and method may be that conventional medical image systems can be easily upgraded to incorporate the features and advantages of the present systems, devices, and methods.

[0109] Of course, it is to be appreciated that any one of the examples, examples or processes described herein may be combined with one or more other examples, examples and / or processes or be separated and / or performed amongst separate devices or device portions in accordance with the present systems, devices and methods.

[0110] Finally, the above-discussion is intended to be merely illustrative of the present systems and methods and should not be construed as limiting the appended claims to any particular example or group of examples. Thus, while the present system has been described in particular detail with reference to exemplary examples, it should also be appreciated that numerous modifications and alternative examples may be devised by those having ordinary skill in the art without departing from the broader and intended spirit and scope of the present systems and methods as set forth in the claims2024PF00292 that follow. Accordingly, the specification and drawings are to be regarded in an illustrative manner and are not intended to limit the scope of the appended claims.

Claims

2024PF00292CLAIMSWhat is claimed is:

1. An ultrasound imaging system (100) comprising: a display (138); and at least one processor (126, 128, 136, 140) configured to: predict (1002) one or more anatomies in a first path (604) of a sweep of a blind sweep ultrasound imaging protocol (600); receive (1004) data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol; make a determination (1006) whether the data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol includes the predicted one or more anatomies; predict (1008) a second path (612) of the sweep of the blind sweep ultrasound imaging protocol, based on the determination; and cause (1010) the second path of the blind sweep ultrasound imaging protocol to be provided on the display.

2. The ultrasound imaging system of claim 1, wherein the at least one processor is further configured to cause the first path to be provided on the display with the second path.

3. The ultrasound imaging system of claim 1, wherein the predicted second path comprises an updated starting point.

4. The ultrasound imaging system of claim 1, further comprising an ultrasound probe configured to acquire the data corresponding to the first path, wherein the at least one processor is further configured to determine a location of the ultrasound probe based, at least in part, on the determination.

5. The ultrasound imaging system of claim 4, wherein the at least one processor is further configured to cause the location of the ultrasound probe to be provided on the display.

6. The ultrasound imaging system of claim 1, further comprising a speaker, wherein the at least one processor is further configured to: cause the speaker to output an audio indication, in response to determining that the data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol does not include the predicted one or more anatomies.2024PF002927. The ultrasound imaging system of claim 1, wherein the at least one processor is further configured to determine the first path of the sweep of the blind sweep ultrasound imaging protocol based on one of an analysis of an image of a subject acquired by a camera or a fundal height measurement of the subject provided by a user.

8. The ultrasound imaging system of any one of the preceding claims, wherein the at least one processor is configured to: predict, using an artificial intelligence model, the one or more anatomies in the first path (604) of the sweep of a blind sweep ultrasound imaging protocol (600); make a determination, using the artificial intelligence model, whether the data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol includes the predicted one or more anatomies; predict, using the artificial intelligence model, the second path (612) of the sweep of the blind sweep ultrasound imaging protocol, based on the determination.

9. A computer-implemented method (1000) of performing a blind sweep protocol, the method comprising: predicting (1002) one or more anatomies in a first path of a sweep of a blind sweep ultrasound imaging protocol; receiving (1004) data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol; determining (1006) whether the data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol includes the predicted one or more anatomies; predicting (1008) a second path of the sweep of the blind sweep ultrasound imaging protocol, based on the determining; and outputting (1010) the second path of the blind sweep ultrasound imaging protocol to a display.

10. The computer- implemented method of claim 9, wherein the blind sweep protocol comprises a plurality of paths including the first and second path, and the method further comprises: shifting remaining ones of the plurality of paths based on the determining; and outputting the second path and the shifted remaining ones of the plurality of paths to the display.

11. The computer-implemented method of claim 9, further comprising: receiving an image from a camera; and determining, using the Al model, a location of an ultrasound probe acquiring the first sweep based on the image.2024PF0029212. The computer- implemented method of claim 11, further comprising outputting a graphic representing the ultrasound probe on the display.

13. The computer- implemented method of claim 9, further comprising determining, using the Al model, a location of an ultrasound probe acquiring the first sweep based on the data corresponding to the first path of the sweep of the blind sweep ultrasound.

14. The computer- implemented method of claim 13, further comprising determining, using the Al model, whether the ultrasound probe is past an expected location of one of the predicted anatomies.

15. The computer-implemented method of claim 14, further comprising outputting an audio indication, a visual indication, or a combination thereof, that the expected location has been passed.

16. The computer-implemented method of claim 9, wherein the blind sweep protocol comprises a plurality of paths, wherein the plurality of paths are determined based, at least in part, on a fundal height provided as an input by a user.

17. The computer-implemented method of any one of claims 9-16, wherein the predicting the one or more anatomies, the determining whether the data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol includes the predicted one or more anatomies and the predicting the second path comprise using an artificial intelligence model.

18. A computer program product comprising instructions that, when executed by a computing system, cause the computing system to: predict (1002) one or more anatomies in a first path of a sweep of a blind sweep ultrasound imaging protocol; receive (1004) data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol; determine (1006) whether the data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol includes the predicted one or more anatomies; predict (1008) a second path of the sweep of the blind sweep ultrasound imaging protocol, based on the determining; and cause (1010) the second path of the blind sweep ultrasound imaging protocol to be provided on the display.2024PF0029219. The computer program product of claim 18, wherein the instructions that, when executed by the computing system, further cause the computing system to cause the first path to be provided on the display with the second path.

20. The computer program product of claim 18, wherein the predicted second path comprises an updated starting point.

21. The computer program product of claim 18, wherein the instructions that, when executed by the computing system, further cause the computing system to determine, using the Al model, whether the ultrasound probe is past an expected location of one or the predicted anatomies.

22. The computer program product of claim 21, wherein the instructions that, when executed by the computing system, further cause the computing system to output an audio indication, a visual indication, or a combination thereof, that the expected location has been passed.

23. The computer program product of any one of claims 18-22, wherein the predicting the one or more anatomies, the determining whether the data corresponding to the first path of the sweep of the blind sweep ultrasound imaging protocol includes the predicted one or more anatomies and the predicting the second path comprise using an artificial intelligence model.28

Images