Systems and methods for automatic motion-mode switching for fetal heart rate determination

Abstract

Apparatuses, systems, and methods for determining heart rate may receive data corresponding to a blind sweep ultrasound protocol by a user performing a blind sweep ultrasound protocol. A heart may be detected, using a neural network, in the data corresponding to the blind sweep ultrasound protocol and the ultrasound imaging system may extract motion-mode (M-mode) signals. At the position of the detected heart, a first M-mode ultrasound signal may be generated, and a heart rate may be determined based on the first M-mode ultrasound signal. The determined heart rate may be displayed to the user. In some cases, a second M-mode signal may be generated at the position of the detected heart and may be used in combination with the first M-mode signal to determine the heart rate.

Description

2024P00294WGSYSTEMS AND METHODS FOR AUTOMATIC MOTION-MODE SWITCHING FOR FETAL HEART RATE DETERMINATIONTECHNICAL FIELD

[0001] The present disclosure relates to acquiring and analyzing medical imaging data. For example, embodiments herein relate to acquiring, analyzing, and calculating various statistics using neural networks.BACKGROUND

[0002] Fetal heart rate detection is an important assessment in antenatal ultrasound imaging. In the current standard of care, trained sonographers may select a region of the fetal heart by hand to acquire motion-mode (M-mode) signals used for determining a fetal heart rate. However, in some maternal care settings, sonographers may not be readily available. Accordingly, it may be difficult to acquire images from the standard planes from which the desired fetal measurements can be obtained. Detecting the fetal heartbeat and obtaining the heart rate may be difficult to preform by a novice ultrasound user, especially when finding a suitable region varies depending on patient characteristics.

[0003] In an attempt to improve access to prenatal imaging, blind sweep ultrasound scans may be performed. These scans may be performed by a healthcare provider with little or no ultrasound imaging training. The healthcare provider may sweep the ultrasound probe in a pre-defined pattern over the patient’s abdomen. Often, the healthcare provider has no access to the images, hence the term “blind sweep.” The ultrasound images may then be provided to a radiologist or other clinician for analysis.

[0004] While blind sweep ultrasound scans may provide clinically useful data, a novice-user who often performs blind sweep ultrasound protocols may not be able to select an accurate region containing the fetal heart while performing the blind sweep ultrasound protocol and thus may not be able to consistently and / or accurately acquire M-mode signals for the determination of a fetal heart rate. As a result, blind sweep ultrasound protocols may not be used to consistently determine a fetal heart rate.SUMMARY

[0005] Apparatuses, systems, and methods for determining heart rate may receive data corresponding to a blind sweep ultrasound protocol by a user performing a blind sweep ultrasound protocol. A heart may be detected, using a neural network, in the data corresponding to the blind sweep ultrasound protocol and the ultrasound imaging system may extract M-mode signals. At the position of the detected heart, a first M-mode ultrasound signal may be generated, and a heart rate may be determinedbased on the first M-mode ultrasound signal. The determined heart rate may be displayed to the user. In some cases, a second M-mode signal may be generated at the position of the detected heart and may be used in combination with the first M-mode signal to determine the heart rate.

[0006] In accordance with at least one example disclosed herein, an ultrasound imaging system is disclosed. The ultrasound imaging system comprises at least one processor configured to receive data corresponding to a blind sweep ultrasound protocol. The at least one processor is further configured to detect, using a neural network, a heart in the data corresponding to the blind sweep ultrasound protocol. The at least one processor is further configured to, in response to detecting the heart, obtain a motionmode (M-mode) signal from the data corresponding to the blind sweep ultrasound protocol at a location where the heart was detected. The at least one processor is further configured to determine a heart rate based on the M-mode ultrasound signal.

[0007] In some embodiments, the at least one processor is further configured to generate a bounding box around the heart, and the M-mode signal passes through the bounding box. In some such embodiments, the at least one processor is further configured to generate a second M-mode signal that passes through the bounding box and determine the heart rate based on an aggregation of the M-mode signal and the second M-mode signal.

[0008] In some embodiments, the ultrasound imaging system further comprises an ultrasound probe for acquiring the data corresponding to the blind sweep ultrasound protocol, wherein the at least one processor is further configured to cause the ultrasound probe to periodically switch between acquiring M-mode data and a brightness-mode (B-mode) data, wherein the at least one processor detects the heart based on the B-mode data and obtains the M-mode signal from the M-mode data.

[0009] In some embodiments, the at least one processor is further configured to determine the heart rate based on counting a number of peaks in the M-mode ultrasound signal over a time interval.

[0010] In some embodiments, the ultrasound imaging system further comprises a speaker, wherein the at least one processor is further configured to cause the speaker to output an audio indication of detection of the heart in the data corresponding to the blind sweep ultrasound protocol.

[0011] In some embodiments, the ultrasound imaging system further comprises a display, wherein the at least one processor is further configured to cause the display to output the determined heart rate to the display.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] FIG. 1 illustrates a block diagram of an ultrasound imaging system arranged in accordance with principles of the present disclosure.2024P00294WG

[0013] FIG. 2 illustrates a flow diagram for estimating a fetal heart rate according to embodiments herein.

[0014] FIG. 3 illustrates examples of generating one or more M-mode signals for determining / calculating a fetal heart rate according to embodiments herein.

[0015] FIG. 4 illustrates a computer-implemented method of determining heart rate according to embodiments herein.DETAILED DESCRIPTION

[0016] The following description of certain examples is illustrative in nature and is in no way intended to limit the disclosed technology or its applications or uses. In the following detailed description of examples of the present apparatuses, systems, and methods, reference is made to the accompanying drawings, which form a part hereof, and in which are shown by way of illustration specific examples in which the described apparatuses, systems, and methods may be practiced. These examples are described in sufficient detail to enable those skilled in the art to practice the presently disclosed apparatuses, systems, and methods, and it is to be understood that other examples may be utilized and that structural and logical changes can be made without departing from the spirit and scope of the present disclosure. Moreover, for the purpose of clarity, detailed descriptions of certain features will not be discussed when they would be apparent to those with skill in the art, so as not to obscure the description of the present technology. The following detailed description is therefore not to be taken in a limiting sense, and the scope of the present technology is defined only by the appended claims.

[0017] Users of medical imaging systems, such as ultrasound imaging systems, face technical challenges related to acquiring images using standard clinical protocols. For example, it may take two or more years to train sonographers to operate ultrasound imaging systems. For example, the sonographer may learn various techniques to obtain clinically useful data, such as techniques to select a region of the fetal heart. The sonographer may also be required to take various measurements either from images (e.g., left ventricle internal diameter from a standard plane image) or other ultrasound data (e.g., blood flow rate from Doppler data or M-mode data). Recently, some ultrasound systems are capable of providing support for sonographers or clinicians with less familiarity with acquiring ultrasound data. For example, a user can perform guided sweeps, in which the user follows guidance provided via a device or system. As a result, the less experienced user may be able to acquire various measurements used for health monitoring or diagnosis.

[0018] In some examples, ultrasound imaging data can be acquired (e.g., by a novice user) by moving an ultrasound probe according to a predetermined grid (termed “blind sweeps”) over a subject (e.g., apregnant subject’s abdomen). Such blind sweep ultrasound scanning protocols are useful, for example in resource-constrained settings or in environments where experienced or trained users of ultrasound imaging systems may not be available. Additionally, blind sweep ultrasound scanning protocols may be useful for acquiring ultrasound imaging data for further processing by one or more machine learning models or other analysis algorithms.

[0019] In the current standard of care, highly trained users (e.g., trained sonographers) may select a region of the fetal heart to acquire the M-mode signals, where the M-mode signals detect motion. If a novice-user is not highly trained the selected region may not be accurate and a fetal heart may not be detected and / or it may not be possible to obtain an accurate measure of the heart rate. For example, the user may perform a blind sweep over a grid pattern including multiple vertical paths and horizontal paths determined based on features of the patient. In some instances, the user may perform a T-sweep blind sweep ultrasound protocol, where the user sweeps over a “T” pattern over the patient's abdomen. In one example, the T-sweep includes a vertical sweep from the fundus of the uterus to the navel, and horizontal sweeps across the abdomen in line with the navel. Alternatively, the vertical sweep may be from the pubic bone to the navel. The T-sweep blind sweep ultrasound protocol may have a high chance of capturing a cardiac signal of the fetus. However, the novice user may not have the skills necessary to extract the M-mode data and analyze it to obtain the fetal heart rate.

[0020] The present disclosure describes a system and related methods for determining a fetal heart rate from ultrasound data acquired from a blind sweep ultrasound protocol or while performing the blind sweep ultrasound protocol. The blind sweep data may acquire ultrasound imaging data, which may include one or more ultrasound images. The images may be included as frames in a cineloop or other data structure. In some embodiments, the images may be B-mode images. A fetal heart may be detected through the use of, for example a neural network (NN), or analysis algorithms of the sort such as artificial intelligence (Al) models and machine learning models. The NN or other Al model may automatically extract M-mode data from the ultrasound data, and calculate the fetal heart rate from the M-mode data. In some embodiments, such as when the fetal is heart is detected during the sweep, the switch to M-mode when a fetal heart is detected to generate M-mode signals from which a fetal heart rate may be determined.

[0021] In some embodiments, a neural network (e.g., a “You Only Look Once” or “YOLO” network or any analysis algorithms of the sort) may detect a position of a heart in data acquired from a blind sweep ultrasound protocol. In some embodiments, the technology may indicate to the user through visual and / or audio indications that the fetal heart has been detected and may prompt the user to keep the probe in its current position (i.e., the position where the fetal heart was detected). The visual and / oraudio indications may be stored in a memory, such as memory and played on a speaker, which may be a part of the user interface, such as user interface.

[0022] The neural network may apply a bounding box to the fetal heart (i.e., encompassing the fetal heart). In some embodiments, the neural network may extract M-mode data from the B-mode data acquired from the blind sweep. The M-mode data may include data from one or more A-lines that pass through the bounding box. In some embodiments, when the fetal heart is detected, the system may automatically switch to acquiring ultrasound data in an M-mode to generate M-mode signals (e.g., when the sweep is initially acquiring B-mode ultrasound data). The M-mode signals may pass through the bounding box of the fetal heart and thus through the fetal heart. The M-mode data may be analyzed to determine / calculate the fetal heart rate. The determined / calculated fetal heart rate may be provided to the user on a display in some embodiments. In some cases, the user may be prompted to continue and finish the blind sweep ultrasound protocol (e.g., to obtain images of other fetal features).

[0023] Alternatively, the system may generate a color doppler signal in place of the M-mode signal and may analyze the color doppler signal to determine / calculate a fetal heart rate. Further, although the examples disclosed herein describe extracting the fetal heart rate from blind sweep data, in other embodiments, the techniques may be applied to data acquired from non-blind sweeps.

[0024] In some implementations, the neural network may detect the fetal heart through segmentation where the neural network may divide the one or more images into multiple segments, where pixels of each segment of the image are associated with an object / type. The neural network may be trained on a large set of ground truth images (e.g., images acquired by blind sweeps or other techniques) that has been annotated by clinically trained experts to accurately identify, for example, the fetal heart or any other feature of interest. The trained neural network may be able to differentiate and identify each object / type (e.g., each feature in the ultrasound protocol data) based in part on being trained on the ground truth data. In some examples, the neural network may apply a bounding box to encompass / surround part of or the entire fetal heart within the blind sweep ultrasound data.

[0025] In some embodiments, if the heart is detected through segmentation by the neural network, the ultrasound system may automatically switch to an M-mode and may generate one or more M-mode signals that may be pass through the bounding box of the detected fetal heart and thus through the fetal heart. The M-mode signals may be automatically analyzed as to determine / calculate the fetal heart rate of the fetal heart. The system may repeatedly insonify the bounding box of the fetal heart or the fetal heart as to achieve a higher frame rate to acquire M-mode data(i.e., higher than the 20 Hz frame rate for B-mode images). In some embodiments, the system may interleave imaging modes. For example, the system may switch from M-mode to the B-mode for a period of time and switch back to the M-mode fora period of time before switching back to B-mode. In some examples, the switching between M-mode and B-mode may ensure the probe has remained in the position of the detected fetal heart and is still detecting the fetal heart. In some embodiments, this may help ensure continuous detection of the fetal heart.

[0026] In some cases, a single line M-mode signal may be generated that passes through the fetal heart. For example, the bounding box applied to the fetal heart within the cineloop may be used to extract a single M-mode signal (i.e., an A-line) that passes through the fetal heart and in some instances passed through the center of the fetal heart which may be analyzed to determine / calculate the fetal heart rate.

[0027] In some other cases, many sectored M-mode signals may be generated and passed through the fetal heart as to determine / calculate the fetal heart rate. This may improve results when an M-mode signal from the center of the bounding box may not provide the best measure of the heart rate. For example, the bounding box may include only part of the fetal heart, or the bounding may be out of orientation (e.g., angled or titled) in comparison to the fetal heart. A stream or multiple M-mode signals may be generated and may pass through the fetal heart detection bounding box at different location and / or from different angles. Generating multiple M-mode signals and passing them through the bounding box of the fetal heart may ensure that the fetal heart rate is accurately measured. In some such cases, an aggregation (e.g., mean, median, or mode) of the calculated fetal heart rate of the M-mode signals may be provided to the user as to provide more accurate results.

[0028] The movement detected by the M-mode data may correspond to the contraction and expansion of the fetal heart. Accordingly, the detected movement may be used to determine the heart rate. Any suitable technique may be used to analyze the M-mode data to determine / calculate the fetal heart rate. In some embodiments, peaks may be automatically extracted / annotated from the M-mode signal using any current method such as discrete wavelet transform (DWT) and / or any machine learning method for time series data (e.g., a long short-term memory (LSTM) model, or a recurrent neural network (RNN)). The fetal heart rate may be imputed from the by calculating the number of peaks corresponding to the same part of the heart signal over a known interval of time. Machine learning protocols may impute the fetal heart rate directly. Optionally, the interval along the M-mode signal line used for analysis may be constrained by the limits of a heart bounding box placed by a neural network or other similar classifiers used in current clinical practice.

[0029] In some embodiments, a same neural network / AI model may be used to detect the heart, extract the M-mode data, and calculate the heart rate from the M-mode data. However, in other embodiments, multiple neural networks / AI models may be used. For example, in some embodiments, a2024P00294WG neural network may detect the heart, and the output may be provided to a second neural network or model that obtains the M-mode data. The output of the second neural network may be provided to third neural network that calculates the heart rate from the M-mode data. Other arrangements may be used in other examples (e.g., a first neural network detects the heart and extracts the M-mode data and a second neural network calculates the heart rate).

[0030] Optionally, an M-mode image (with or without signal peaks labeled), may be shown on a display to the user post-acquisition to be used for clinical explainability and detectability of the fetal heart and fetal heart rate. In some implementations, the system may output to the user a binary indication (e.g., visual and / or audible) if a fetal heart has been detected or not without determining / calculating the fetal heart rate. This may be used by the user as an indication whether to keep performing the blind sweep ultrasound protocol or to stop the probe in its current position as the fetal heart has been detected.

[0031] If the system does not detect a fetal heart after completing the blind sweep, the system may prompt the user to repeat the blind sweep ultrasound protocol. If the fetal heart is still not detected by the system after repeating the blind sweep ultrasound protocol multiple times (e.g., twice, three times, four times...) the user may be prompted to, for example perform a different ultrasound protocol (i.e., not a blind sweep protocol). In some such cases, the user may be prompted to recommend the patient be seen by another healthcare provider (e.g., trained sonographer, obstetrician). It should be understood that not detecting a fetal heart in the performance of a blind sweep ultrasound protocol may not necessarily mean that no fetal heart or fetal heart rate exists. It may be that the signal is too weak to detect a fetal heart or there is signal interference.

[0032] Advantages of the disclosed technology may include introducing systems(s) methods(s) to determine / calculate a fetal heart rate while performing blind sweep ultrasound protocols as currently there is no way for a novice-user to accurately determine / calculate fetal heart rate from a blind sweep ultrasound protocol.

[0033] It should be understood that while examples herein relate to determining / calculating a fetal heart rate using blind sweep ultrasound scanning techniques, it should be appreciated that the disclosed technology may be used to determine / calculate other statistics of interest pertaining to the heart and / or cardiovascular system (e.g., used for cardiovascular applications such as blood flow) and additionally may be used to determine / calculate the heart rate of adults and patients of all ages.

[0034] FIG. 1 illustrates a block diagram of an ultrasound imaging system 100 arranged in accordance with principles of the present disclosure. In the ultrasound imaging system 100 of FIG. 1, an ultrasound probe 112 includes a transducer array 114 for transmitting ultrasonic waves and receivingecho information. The transducer array 114 can be implemented as a linear array, convex array, a phased array, and / or a combination thereof. The transducer array 114, for example, can include a two- dimensional array (as shown) of transducer elements capable of scanning in both elevation and azimuth dimensions for 2D and / or 3D imaging. The transducer array 114 can be coupled to a microbeamformer 116 in the probe 112, which controls transmission and reception of signals by the transducer elements in the array. In this example, the microbeamformer 116 is coupled by the probe cable to a transmit / receive (T / R) switch 118, which switches between transmission and reception and protects the main beamformer 122 from high-energy transmit signals. In some embodiments, the T / R switch 118 and other elements in the system can be included in the ultrasound probe 112 rather than in a separate ultrasound system base. In some embodiments, the ultrasound probe 112 may be coupled to the ultrasound imaging system via a wireless connection (e.g., WiFi, Bluetooth).

[0035] The transmission of ultrasonic beams from the transducer array 114 under control of the microbeamformer 116 is directed by the transmit controller 120 coupled to the T / R switch 118 and the beamformer 122, which receives input from the user’s operation of the user interface (e.g., control panel, touch screen, console) 125. The user interface 125 may include soft and / or hard controls. One of the functions controlled by the transmit controller 120 is the direction in which beams are steered. Beams may be steered straight ahead from (orthogonal to) the transducer array 114, or at different angles for a wider field of view. The partially beamformed signals produced by the microbeamformer 116 are coupled via channels 115 to a main beamformer 122 where partially beamformed signals from individual patches of transducer elements are combined into a fully beamformed signal. In some embodiments, microbeamformer 116 is omitted and the transducer array 114 is coupled via channels 115 to the beamformer 122. In some embodiments, the system 100 can be configured (e.g., include a sufficient number of channels 115 and have a transmit / receive controller programmed to drive the transducer array 114) to acquire ultrasound data responsive to a plane wave or diverging beams of ultrasound transmitted toward the subject. In some embodiments, the number of channels 115 from the ultrasound probe may be less than the number of transducer elements of the transducer array 114 and the system can be operable to acquire ultrasound data packaged into a smaller number of channels than the number of transducer elements.

[0036] The beamformed signals are coupled to a signal processor 126. The signal processor 126 can process the received echo signals in various ways, such as bandpass filtering, decimation, I and Q component separation, and / or harmonic signal separation. The signal processor 126 can also perform additional signal enhancement such as speckle reduction, signal compounding, and noise elimination. The processed signals are coupled to a B-mode processor 128, which can employ amplitude detectionfor the imaging of structures in the body. The signals produced by the B-mode processor 128 are coupled to a scan converter 130 and a multiplanar reformatter 132. The scan converter 130 arranges the echo signals in the spatial relationship from which they were received in a desired image format. For instance, the scan converter 130 can arrange the echo signal into a two-dimensional (2D) sector-shaped format, or a pyramidal three-dimensional (3D) image. The multiplanar reformatter 132 can convert echoes, which are received from points in a common plane in a volumetric region of the body into an ultrasonic image of that plane, as described in U.S. Pat. No. 6,443,896 (Detmer).

[0037] A volume Tenderer 134 converts the echo signals of a 3D data set into a projected 3D image as viewed from a given reference point, e.g., as described in U.S. Pat. No. 6,530,885 (Entrekin et al.). In some implementations, the system 100 can additionally or alternatively be configured to perform 3D and / or 4D acquisitions, such as protocolized 3D / 4D acquisitions. The 2D or 3D images can be coupled from the scan converter 130, multiplanar reformatter 132, and volume Tenderer 134 to at least one processor 137 for further image processing operations. For example, the at least one processor 137 can include an image processor 136 configured to perform further enhancement and / or buffering and temporary storage of imaging data for display on an image display 138. The display 138 can include a display device implemented using a variety of display technologies, such as ECD, LED, OLED, or plasma display technology. The at least one processor 137 can include a graphics processor 140, which can generate graphic overlays for display with the ultrasound images. These graphic overlays can contain, e.g., standard identifying information such as patient name, date and time of the image, imaging parameters, and the like. For these purposes the graphics processor 140 receives input from the user interface 125, such as a typed patient name. The user interface 125 can also be coupled to the multiplanar reformatter 132 for selection and control of a display of multiple multiplanar reformatted (MPR) images.

[0038] The user interface 125 can include one or more mechanical controls, such as buttons, dials, a trackball, a physical keyboard, and others, which may also be referred to herein as hard controls. Alternatively, or additionally, the user interface 125 can include one or more soft controls, such as buttons, menus, soft keyboard, and other user interface control elements implemented for example using touch-sensitive technology (e.g., resistive, capacitive, or optical touch screens). One or more of the user controls can be co-located on a control panel 124. For example, one or more of the mechanical controls can be provided on a console and / or one or more soft controls can be co-located on a touch screen, which can be attached to or integral with the console. The display 138 and the user interface 125 can be included in an I / O component, via which outputs are provided by the system 100 and / or inputs are received by the system 100.2024P00294WG

[0039] In some implementations, the user interface 125 can receive inputs and provide outputs of the disclosed system. For example, the user interface 125 can receive a user input specifying a T-sweep workflow or any blind sweep ultrasound protocol.

[0040] Although described as separate processors, it will be understood that the functionality of any of the processors described herein can be implemented in a single processor (e.g., a CPU or GPU implementing the functionality of processor 137) or fewer number of processors than described in this example. In some embodiments, the at least one processor 137 can be hardware-based (e.g., include multiple layers of interconnected nodes implemented in hardware). In some embodiments, the at least one processor 137 can be implemented at other processing stages, e.g., prior to the processing performed by the image processor 136, volume Tenderer 134, multiplanar reformatter 132, and / or scan converter 130. In some embodiments, the at least one processor 137 can be implemented to process ultrasound data in the channel domain, beamspace domain (e.g., before or after beamformer 122), the IQ domain (e.g., before, after, or in conjunction with signal processor 126), and / or the k-space domain. As described, in some embodiments, functionality of two or more of the processing components (e.g., beamformer 122, signal processor 126, B-mode processor 128, scan converter 130, multiplanar reformatter 132, volume Tenderer 134, at least one processor 137, image processor 136, graphics processor 140, etc.) can be combined into a single processing unit and / or divided between multiple processing units. The processing units can be implemented in software, hardware, or a combination thereof. For example, the at least one processor 137 can include one or more graphical processing units (GPU). In another example, beamformer 122 can include an application specific integrated circuit (ASIC).

[0041] The at least one processor 137 can be coupled to one or more computer-readable media (e.g., memory 142) included in the system 100, which can be non-transitory. The one or more computer- readable media can carry instructions and / or a computer program that, when executed, cause the at least one processor 137 to perform operations described herein. For example, the memory 142 may include instructions for implementing one or more neural networks (or other Al model) for detecting a fetal heart from ultrasound data, generating / extracting M-mode data, and / or calculating a fetal heart rate. A computer program can be stored / distributed on any suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, and can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Furthermore, embodiments can take the form of a computer program product accessible from a computer-readable medium providing program code for use by or in connection with a computer or any device or system that executes instructions. For the purposes of this disclosure, a computer-readable medium can generally be any tangible apparatus that can contain, store, communicate, propagate, and / or transport the program for use by or in connection with the instruction execution device. The computer-readable medium can be, for example, without limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, and / or a propagation medium. Non-limiting examples of a computer readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk, and / or an optical disk. Optical disks can include compact disk read only memory (CD-ROM), compact disk-read / write (CD-R / W), and / or DVD.

[0042] In some embodiments, while shown separately, many of the components of system 100 may be included within the probe 112. In some embodiments, some of the components of system 100 may be included in a tablet or other portable computing device. In some applications, this may provide an ultrasound imaging system that may be more portable and / or more suitable for care in remote / rural environments compared to traditional cart-based ultrasound systems. An example of such a portable system is the Philips® Lumify® system. However, this is merely an example, and the disclosure is not limited to this specific system.

[0043] The system 100 may be capable of performing the techniques disclosed herein for detecting the fetal heart in blind sweep ultrasound data, extracting the M-mode signal, and calculating the fetal heart rate therefrom according to principles of the present disclosure. However, in some instances, the system 100 may acquire the blind sweep data, and that data may be provided to a computing system with one or more processors (e.g., signal processor, image processor, graphics processor, multiplanar reformatter, volume Tenderer), a non-transitory computer readable medium, and a user interface. The computing system may perform some or all of the remaining non-image acquisition functions in these embodiments. Accordingly, while the examples provided herein describe data analysis functions being performed by an ultrasound imaging system, such as system 100, the disclosure is not limited to these embodiments.

[0044] FIG. 2 illustrates a flow diagram 200 for estimating a fetal heart rate according to embodiments herein.

[0045] The flow diagram 200 begins with the user performing a blind sweep ultrasound protocol (e.g., a grid based blind sweep ultrasound protocol or T-sweep based blind sweep ultrasound protocol) on the patient’s abdomen using an ultrasound probe at block 202. In some embodiments, the probe may be probe 112 shown in FIG. 1.

[0046] At block 204, the data corresponding to the blind sweep ultrasound protocol may be analyzed to detect a fetal heart. In some embodiments, a neural network (e.g., YOLO network, other Al model, orany analysis algorithm of the sort), may be used to detect the fetal heart in the data. In some examples, this may be accomplished through segmentation of the fetal heart by the neural network or other algorithm. In some examples, the neural network may apply a bounding box to the fetal heart in the data corresponding to the blind sweep ultrasound protocol. In some embodiments, the neural network and / or other analysis algorithm may be implemented by one or more processors, such as the image processor 136.

[0047] In some examples, if the fetal heart is detected, the user may be prompted by the system to stop the probe at the position where the fetal heart was detected at block 206. In some examples, the system may perform an automatic mode switch to an M-mode (or alternatively a color doppler mode) for heartbeat detection at block 208. At block 210, the system may generate an M-mode signal (or alternatively a color doppler signal) that may be passed through the detected heart. In some cases, a single M-mode signal may be generated as to pass through the detected fetal heart. In some other cases, more than one M-mode signals may be generated as to pass through the detected fetal heart. In some examples, the system may periodically switch between the M-mode and a B-mode as to achieve a higher frame rate. In some embodiments, the M-mode data may be extracted from the B-mode data acquired by the blind sweep.

[0048] If no fetal heart is detected after completion of the blind sweep ultrasound protocol, the system may prompt the user to repeat the blind sweep ultrasound protocol. If still no fetal heart is detected the system may recommend for the patient to see another healthcare provider or for the user to perform a different ultrasound protocol for determining the fetal heart rate . The prompts may be an audio indication provided on a speaker and / or a visual indication (e.g., graphic, text) provided on a display, such as display 138.

[0049] At block 212, the resulting M-mode signal(s) from the detection of the fetal heart may be extracted. Optionally, the M-mode signal(s) may be plotted over time illustrating the motion of the fetal heart including various peaks over time and provided on the display. At block 214, the M-mode signal(s) may be analyzed to determine / calculate the fetal heart rate by, for example counting the number peaks of the fetal heart rate over a certain period of time. Various techniques such as DWT, RNN, and / or LSTM may be used to determine the fetal heart rate based on the M-mode signal. In embodiments where multiple M-mode signals were generated / extracted, the fetal heart rate may be an average and / or median of the heart rates calculated from the multiple M-mode signals.

[0050] At block 216, the determined / calculated fetal heart rate may be provided to the user on a display, for example display 138. In some embodiments, the display may be on a tablet, which includes at least some components of an ultrasound imaging system, such as system 100 In some embodiments,2024P00294WG additional information such as images of the detected fetal heart or acquired M-mode data may be displayed. In some embodiments, after the fetal heart rate is determined / calculated, the user may be prompted by the system to continue performing the blind sweep ultrasound protocol.

[0051] FIG. 3 illustrates examples of generating one or more M-mode signals for determining / calculating a fetal heart rate according to embodiments herein.

[0052] As discussed, in reference to FIG. 2, a bounding box 308 may be applied to the detected fetal heart 310. In some cases, such as case 302, a single M-mode signal 306 may be generated through a center of the bounding box 308.When the detected fetal heart 310 is centered (or approximately centered) in the bounding box 308, the M-mode signal 306 passes through the detected fetal heart 310. The single M-mode signal 306 may be used for a determination / calculation of the fetal heart rate.

[0053] In some other cases, such as case 304, the fetal heart 310 may not be centered and / or completely encompassed by the bounding box 308. In these cases, applying a single M-mode signal 306 as in case 302 may not be sufficient for obtaining an accurate fetal heart rate. Accordingly, in some embodiments, multiple M-mode signals 306 may be generated through the bounding box 308. In the example shown in case 304, four M-mode signals 306A-D are generated that pass through the bounding box 308 of the fetal heart 310. However, in other examples, more or fewer M-mode signals 306 may be generated. Generating multiple M-mode signals 306A-D may ensure that a more accurate fetal heart rate is detected. In some instances, the resulting three fetal heart rates from the M-mode signals 306A- D may be aggregated (e.g., mean, median, or mode) for a fetal heart rate determination / calculation.

[0054] However, in some embodiments, only some of the M-mode signals 306A-D may be used. For example, M-mode signal 306D does not pass through the fetal heart 310. Accordingly, no heart rate may be detected or a “heart rate” (e.g., noise / motion artifacts) having an amplitude and / or frequency outside an expected range may be detected. Thus, in some embodiments, M-mode signals 306 resulting in heart rates outside one or more ranges (e.g., amplitude, frequency) may be excluded from aggregation with the other M-mode signals 306.

[0055] FIG. 4 illustrates a computer-implemented method of determining heart rate according to embodiments herein. The computer-implemented method 400 may be implemented in whole or in part by an ultrasound imaging system, such as system 100 in some examples. In other examples, the computer-implemented method 400 may be implemented in whole or in part by a computing system, such as a radiology workstation, that may not have ultrasound imaging capabilities. The illustrated method 400 includes receiving data corresponding to a blind sweep ultrasound protocol as indicated by block 402. For example, the data corresponding to the blind sweep ultrasound protocol may be stored in memory 142, and the processor 137 may receive the data from the memory 142. In some examples, the2024P00294WG data corresponding to the blind sweep ultrasound protocol may be received during performance of the blind sweep ultrasound protocol as illustrated at block 202.

[0056] The method 400 further includes detecting, using a neural network, a heart in the data corresponding to the blind sweep ultrasound protocol as indicated by block 404. For example, the neural network at block 204 may be implemented by image processor 136 and / or graphics processor 140 to detect a heart in the data corresponding to the blind sweep ultrasound protocol.

[0057] The method 400 further includes in response to detecting the heart, obtaining an M-mode signal from the data corresponding to the blind sweep ultrasound protocol at a location where the heart was detected as indicated by block 406. For example, an M-mode signal may be obtained by the image processor 136 and / or graphics processor 140 from data corresponding to the blind sweep ultrasound protocol. For example, as illustrated at block 210 of FIG. 2. In some other examples, at block 406 multiple M-mode signals may be obtained from data corresponding to the blind sweep ultrasound protocol at the location where the heart was detected as illustrated in case 304 of FIG. 3.

[0058] The method 400 further includes determining a heart rate based on the M-mode ultrasound signal at block 408. For example, a heart rate may be determined by the image processor 136, graphics processor 140, and / or signal processor 126. In some examples, determining the heart rate may be based on counting a number of peaks in the M-mode ultrasound signal over a time interval. In some examples, block 408 may be performed by image processor 136 and / or graphics processor 140 to count a number of peaks in the M-mode ultrasound signal over a time interval.

[0059] In some embodiments, the method 400 further comprises generating a bounding box around the heart, and the M-mode signal passes through the bounding box. Some such embodiments further comprise generating a second M-mode signal that passes through the bounding box and determine the heart rate based on an aggregation of the M-mode signal and the second M-mode signal. For example, the neural network at block 204 may be implemented by image processor 136 and / or graphics processor 140 to generate a bounding box around the heart.

[0060] In some embodiments, the method 400 further comprises causing an ultrasound probe, such as ultrasound probe 112, to periodically switch between acquiring M-mode data and a brightness-mode (B-mode) data, wherein the at least one processor detects the heart based on the B-mode data and obtains the M-mode signal from the M-mode data. For example, the ultrasound probe may switch between acquiring M-mode data and B-mode data through the use of the beamformer 122, the signal processor 126, and / or the B-mode processor 128.

[0061] In some embodiments, the method 400 further comprises causing a speaker to output an audio indication of detection of the heart in the data corresponding to the blind sweep ultrasound protocol. For example, at block 404 a speaker may output an audio indication of detection of the heart.

[0062] In some embodiments, the method 400 further comprises causing a display to output the determined heart rate to the display. For example, the display 138 may output the determined heart rate calculated at block 408.

[0063] The systems, methods, and apparatuses disclosed herein may allow for automatic fetal heart rate detection. In some embodiments, this may be achieved by detection of the fetal heart from blind sweep ultrasound data and extracting and / or generating M-mode data. The M-mode data may be analyzed to provide the fetal heart rate. The systems, methods, and apparatuses disclosed herein may allow for fetal heart rate measurements by non-skilled ultrasound users. This may allow improved access to maternal healthcare in areas where trained sonographers are limited.

[0064] In various examples where components, systems and / or methods are implemented using a programmable device, such as a computer-based system or programmable logic, it should be appreciated that the above-described systems and methods can be implemented using any of various known or later developed programming languages, such as “Python”, “C”, “C++”, “FORTRAN”, “Pascal”, “VHDL” and the like. Accordingly, various storage media, such as magnetic computer disks, optical disks, electronic memories and the like, can be prepared that can contain information that can direct a device, such as a computer, to implement the above-described systems and / or methods. Once an appropriate device has access to the information and programs contained on the storage media, the storage media can provide the information and programs to the device, thus enabling the device to perform functions of the systems and / or methods described herein. For example, if a computer disk containing appropriate materials, such as a source file, an object file, an executable file or the like, were provided to a computer, the computer could receive the information, appropriately configure itself and perform the functions of the various systems and methods outlined in the diagrams and flowcharts above to implement the various functions. That is, the computer could receive various portions of information from the disk relating to different elements of the above-described systems and / or methods, implement the individual systems and / or methods and coordinate the functions of the individual systems and / or methods described above.

[0065] In view of this disclosure it is noted that the various methods and devices described herein can be implemented in hardware, software, and / or firmware. Further, the various methods and parameters are included by way of example only and not in any limiting sense. In view of this disclosure, those of ordinary skill in the art can implement the present teachings in determining their own techniques andneeded equipment to affect these techniques, while remaining within the scope of the invention. The functionality of one or more of the processors described herein may be incorporated into a fewer number or a single processing unit (e.g., a CPU) and may be implemented using application specific integrated circuits (ASICs) or general-purpose processing circuits which are programmed responsive to executable instructions to perform the functions described herein.

[0066] Although the present system may have been described with particular reference to an ultrasound imaging system, it is also envisioned that the present system can be extended to other medical imaging systems where one or more images are obtained in a systematic manner. Accordingly, the present system may be used to obtain and / or record image information related to, but not limited to renal, testicular, breast, ovarian, uterine, thyroid, hepatic, lung, musculoskeletal, splenic, cardiac, arterial and vascular systems, as well as other imaging applications related to ultrasound-guided interventions. Further, the present system may also include one or more programs which may be used with conventional imaging systems so that they may provide features and advantages of the present system. Certain additional advantages and features of this disclosure may be apparent to those skilled in the art upon studying the disclosure, or may be experienced by persons employing the novel system and method of the present disclosure. Another advantage of the present systems and method may be that conventional medical image systems can be easily upgraded to incorporate the features and advantages of the present systems, devices, and methods.

[0067] Of course, it is to be appreciated that any one of the examples, examples or processes described herein may be combined with one or more other examples, examples and / or processes or be separated and / or performed amongst separate devices or device portions in accordance with the present systems, devices and methods.

[0068] Finally, the above-discussion is intended to be merely illustrative of the present systems and methods and should not be construed as limiting the appended claims to any particular example or group of examples. Thus, while the present system has been described in particular detail with reference to exemplary examples, it should also be appreciated that numerous modifications and alternative examples may be devised by those having ordinary skill in the art without departing from the broader and intended spirit and scope of the present systems and methods as set forth in the claims that follow. Accordingly, the specification and drawings are to be regarded in an illustrative manner and are not intended to limit the scope of the appended claims.

Claims

CLAIMSWhat is claimed is:

1. An ultrasound imaging system comprising: at least one processor configured to: receive data corresponding to a blind sweep ultrasound protocol; detect, using a neural network, a heart in the data corresponding to the blind sweep ultrasound protocol; in response to detecting the heart, obtain a motion-mode (M-mode) signal from the data corresponding to the blind sweep ultrasound protocol at a location where the heart was detected; and determine a heart rate based on the M-mode ultrasound signal.

2. The ultrasound imaging system of claim 1, the processor is further configured to generate a bounding box around the heart, and the M-mode signal passes through the bounding box.

3. The ultrasound imaging system of claim 2, wherein the at least one processor is further configured to generate a second M-mode signal that passes through the bounding box and determine the heart rate based on an aggregation of the M-mode signal and the second M-mode signal.

4. The ultrasound imaging system of claim 1, wherein the ultrasound imaging system further comprises an ultrasound probe for acquiring the data corresponding to the blind sweep ultrasound protocol, wherein the at least one processor is further configured to cause the ultrasound probe to periodically switch between acquiring M-mode data and a brightness-mode (B-mode) data, wherein the at least one processor detects the heart based on the B-mode data and obtains the M-mode signal from the M-mode data.

5. The ultrasound imaging system of claim 1, wherein the at least one processor is further configured to determine the heart rate based on counting a number of peaks in the M-mode ultrasound signal over a time interval.

6. The ultrasound imaging system of claim 1, further comprising a speaker, wherein the at least one processor is further configured to: cause the speaker to output an audio indication of detection of the heart in the data corresponding to the blind sweep ultrasound protocol.

7. The ultrasound imaging system of claim 1, further comprising a display, wherein the at least one processor is further configured to cause the display to output the determined heart rate to the display.

8. A computer-implemented method of determining heart rate, the method comprising: receiving data corresponding to a blind sweep ultrasound protocol; detecting, using a neural network, a heart in the data corresponding to the blind sweep ultrasound protocol; in response to detecting the heart, obtaining a motion-mode (M-mode) signal from the data corresponding to the blind sweep ultrasound protocol at a location where the heart was detected; and determining a heart rate based on the M-mode ultrasound signal.

9. The computer-implemented method of claim 8, further comprising generating a bounding box around the heart, and the M-mode signal passes through the bounding box.

10. The computer-implemented method of claim 9, further comprising generating a second M-mode signal that passes through the bounding box and determine the heart rate based on an aggregation of the M-mode signal and the second M-mode signal.

11. The computer-implemented method of claim 8, further comprising causing an ultrasound probe to periodically switch between acquiring M-mode data and a brightness-mode (B-mode) data, and wherein the computer-implemented method further comprises detecting the heart based on the B-mode data and obtains the M-mode signal from the M-mode data.

12. The computer-implemented method of claim 8, further comprising determining the heart rate based on counting a number of peaks in the M-mode ultrasound signal over a time interval.

13. The computer-implemented method of claim 12, wherein determining the heart rate based on counting the number of peaks in the M-mode ultrasound signal over the time interval further comprises using a discrete wavelet transform (DWT), a long short-term memory (LSTM) model, a recurrent neural network (RNN), or a combination thereof.

14. The computer-implemented method of claim 8, further comprising causing a display to output the determined heart rate to the display.

15. The computer-implemented method of claim 8, wherein obtaining the M-mode signal from the data corresponding to the blind sweep ultrasound protocol further comprises using a second neural network,and wherein determining the heart rate based on the M-mode ultrasound signal further comprises using a third neural network.

16. At least one non-transitory computer-readable medium carrying instructions that, when executed by a computing system, cause the computing system to: receive data corresponding to a blind sweep ultrasound protocol; detect, using a neural network, a heart in the data corresponding to the blind sweep ultrasound protocol; in response to detecting the heart, obtain a motion-mode (M-mode) signal from the data corresponding to the blind sweep ultrasound protocol at a location where the heart was detected; and determine a heart rate based on the M-mode ultrasound signal.

17. The non-transitory computer-readable medium of claim 16, wherein the instructions that, when executed by the computing system, further cause the computing system to generate a bounding box around the heart, and the M-mode signal passes through the bounding box.

18. The non-transitory computer-readable medium of claim 17, wherein the instructions that, when executed by the computing system, further cause the computing system to generate a second M-mode signal that passes through the bounding box and determine the heart rate based on an aggregation of the M-mode signal and the second M-mode signal.

19. The non-transitory computer-readable medium of claim 16, wherein the instructions that, when executed by the computing system, further cause the computing system to determine the heart rate based on counting a number of peaks in the M-mode ultrasound signal over a time interval.

20. The non-transitory computer-readable medium of claim 16, wherein the instructions that, when executed by the computing system, further cause the computing system to cause an ultrasound probe to periodically switch between acquiring M-mode data and a brightness-mode (B-mode) data, wherein the at least one processor detects the heart based on the B-mode data and obtains the M-mode signal from the M-mode data.

Images