Techniques for determining the position of a fetus during an imaging procedure

Abstract

Techniques are disclosed for determining a fetal position during an ultrasound imaging procedure. The techniques can include obtaining an ultrasound image of a fetus within the uterus. The fetus will include a torso and a spine, and the ultrasound image can include a circumferential view of the torso of the fetus. The circumferential view can include at least: (i) an outer boundary of the torso, and (ii) the spine including three landmarks arranged in a triangular orientation. The techniques can also include: superimposing a fetal overlay based on alignment instructions corresponding to an alignment between the outer boundary of the torso and the three landmarks to obtain an augmented reality image of the fetus. The fetal overlay can include graphical elements indicating the left and right sides of the fetus. The augmented reality image of the fetus can be output on a display of an ultrasound machine.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims the benefit of U.S. Provisional Application Serial No. 62 / 898,932, filed on September 11, 2019. The disclosure of the above application is hereby incorporated by reference in its entirety. Technical Field

[0003] The present disclosure generally relates to improved fetal imaging procedures, and more particularly to techniques for easily determining and monitoring the position of a fetus during an ultrasound imaging procedure of the fetus. Background Art

[0004] The background description provided herein is for the purpose of generally presenting the background of the present disclosure. Neither expressly nor implicitly admitted is that the work of the presently named inventors within the scope described in this background section and aspects that may otherwise not conform to the prior art description at the time of filing are prior art to the present disclosure.

[0005] Fetal ultrasound is an imaging technique commonly used during pregnancy to monitor fetal development. Among them, a typical fetal ultrasound procedure will examine the internal organs of the fetus to screen for any abnormalities. For example, a fetal ultrasound procedure can screen for situs inversus of the fetus, that is, a situation where some internal organs (heart, stomach, liver, etc.) in the fetal chest / abdomen are located on the opposite side of their normal positions. In some cases, doctors may wish to make special arrangements to assist in the delivery and initial care of an infant diagnosed with in - utero situs inversus. Therefore, an ultrasound technician / sonographer can determine the position of such internal organs during the ultrasound procedure, which is sometimes referred to as "fetal position".

[0006] Current techniques for determining fetal position during an ultrasound procedure are complex, confusing, and difficult to handle. Experienced sonographers can use different techniques to determine fetal position, and it is not uncommon for two experienced sonographers to disagree on the fetal position of a particular fetus based on such different techniques. Therefore, there is a need for improved techniques for determining fetal position during an ultrasound procedure. Summary of the Invention

[0007] According to various aspects of the present disclosure, a computer-implemented method for determining the position of a fetus during an ultrasound imaging procedure is disclosed. The method may include: obtaining an ultrasound image of a fetus in utero at an ultrasound machine having a display and at least one processor. The fetus will include a torso and a spine, and the ultrasound image may include a circumferential view of the torso of the fetus. The circumferential view may include at least: (i) the outer boundary of the torso and (ii) the spine including three landmarks arranged in a triangular orientation. The method may further include: superimposing a fetal overlay at the ultrasound machine based on alignment instructions corresponding to the alignment between the outer boundary of the torso and the three landmarks arranged in a triangular orientation to obtain an augmented reality image of the fetus. The fetal overlay may include graphical elements indicating the left and right sides of the fetus. The augmented reality image of the fetus may be output on the display of the ultrasound machine.

[0008] In some implementations, the alignment instructions for aligning the fetal overlay with the outer boundary of the torso and the three landmarks arranged in a triangular orientation may be received at the ultrasound machine via user input. In additional or alternative implementations, the alignment instructions may be generated by the ultrasound machine based on the alignment between the outer boundary of the torso and the three landmarks arranged in a triangular orientation. In such implementations, the method may further include: detecting movement of the fetus in the ultrasound image at the ultrasound machine; and adjusting the fetal overlay at the ultrasound machine based on the movement of the fetus such that the graphical elements correctly indicate the left and right sides of the fetus in the augmented reality image of the fetus. Adjusting the fetal overlay based on the movement of the fetus may be based on the alignment between the outer boundary of the torso and the three landmarks arranged in a triangular orientation.

[0009] In other aspects, the method may further include: outputting a graphical representation of the fetus on the display of the ultrasound machine, wherein the graphical representation depicts the orientation of the fetus relative to the augmented reality image of the fetus. As the user changes the position of the image, the orientation of the graphical representation of the fetus may change.

[0010] In some implementations, the method may further include: determining an intermediate position of the augmented reality image of the fetus at the ultrasound machine; and outputting an intermediate position indicator on the display of the ultrasound machine when the augmented reality image of the fetus is at the intermediate position.

[0011] In some implementations, the method may further include: receiving an instruction to change the position of the augmented reality image at the ultrasound machine; and in response to receiving the instruction to change the position, adjusting the fetal overlay at the ultrasound machine such that the graphical elements correctly indicate the left and right sides of the fetus in the augmented reality image of the fetus. Additionally, in some implementations, the graphical elements indicating the left and right sides of the fetus may include at least one translucent color indicator.

[0012] In other implementations, the present disclosure relates to an ultrasound machine including a display and at least one processor. The ultrasound machine can perform any, any combination, or all of the methods described above.

[0013] Based on the detailed description provided below, other application areas of the present disclosure will become apparent. It should be understood that the detailed description and specific examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The present disclosure will be more fully understood from the detailed description and the accompanying drawings, in which:

[0015] Figure 1 is an illustration of an exemplary apparatus of an ultrasound machine and a patient for an obstetric ultrasound procedure in accordance with some aspects of the present disclosure;

[0016] Figure 2 is a functional block diagram of an exemplary ultrasound machine in accordance with some aspects of the present disclosure;

[0017] Figure 3 is Figure 2 an illustration of an exemplary ultrasound probe of an exemplary ultrasound machine;

[0018] Figure 4 is a representation of a display device of an exemplary ultrasound machine when a graphical user interface is being displayed during an obstetric ultrasound procedure in accordance with some implementations of the present disclosure Figure 2 ;

[0019] Figures 5A to 5C is a representation of a display device of an exemplary ultrasound machine when various graphical user interfaces are being displayed during an obstetric ultrasound procedure in accordance with some implementations of the present disclosure Figure 2 ;

[0020] Figure 5D is a representation of a display device of an exemplary ultrasound machine when another graphical user interface is being displayed during an obstetric ultrasound procedure in accordance with some implementations of the present disclosure Figure 2 ;

[0021] Figure 6 is a representation of a display device of an exemplary ultrasound machine when yet another graphical user interface is being displayed during an obstetric ultrasound procedure in accordance with some implementations of the present disclosure Figure 2 ;

[0022] Figure 7 is a representation of a display device of an exemplary ultrasound machine when another graphical user interface is being displayed during an obstetric ultrasound procedure in accordance with some implementations of the present disclosure Figure 2 ;

[0023] Figure 8A and Figure 8B are illustrations of fetal overlays that, according to some implementations of the present disclosure, are to be superimposed on an ultrasound image to obtain an augmented reality image of a fetus;

[0024] Figure 9 is a representation of a display device of an example ultrasound machine when a particular graphical user interface is being displayed during an obstetric ultrasound procedure Figure 2 according to some implementations of the present disclosure; and

[0025] Figure 10 is a flowchart of an example technique for determining a fetus's position during an imaging procedure according to some implementations of the present disclosure. DETAILED DESCRIPTION

[0026] An ultrasound technician can capture hundreds of images and videos during an ultrasound procedure. To directly image a patient, the ultrasound technician will have a general idea of the location of the part of the patient being imaged based on the position of the ultrasound probe relative to the patient. Just by way of example, when the ultrasound probe is placed on the right side of the patient's body, the ultrasound technician will know or be able to easily confirm that the right side of her / his body is being imaged. However, with fetal ultrasound, since the fetus can be oriented three-dimensionally within the patient, it may not be easy to determine the part of the fetus being imaged. Additionally, the fetus may change position during the ultrasound procedure. As described more fully herein, it may be important for an ultrasound technician to know the position and orientation of the fetus within the patient for various aspects of the ultrasound procedure, including but not limited to determining fetal laterality and fetal position.

[0027] As briefly mentioned above, current techniques for determining a fetus's position during an ultrasound procedure are complex, confusing, and difficult to handle, and it is not uncommon for even experienced sonographers to make mistakes. Accordingly, the present disclosure relates to improved techniques for determining a fetus's position. More specifically, the present disclosure describes techniques for determining the left and right sides of a fetus during an imaging procedure and generating an augmented reality image of the fetus to be displayed to the sonographer. The augmented reality image includes a fetal overlay superimposed on the fetal image. The fetal overlay can include graphical elements indicating the left and right sides of the fetus. In an example implementation, the graphical elements can include translucent color indicators that "highlight" and distinguish the different sides of the fetus, such as presenting the left side of the fetus in green and the right side of the fetus in red. Other implementations of the graphical elements are also within the scope of the present disclosure.

[0028] Align the fetal overlay with the fetus in the fetal image based on an alignment instruction. The alignment instruction can correspond to an alignment between two or more landmarks of the fetus. In an example implementation, the torso and spine of the fetus can be used to generate the alignment instruction. By way of example only, in a circumferential view of the fetus at certain positions, the fetal image includes a view of the outer boundary of the fetus's torso and the spine. More specifically, and as further described below, the spine can appear to have three landmarks, arranged in a triangular orientation and as circles or "points" near the outer boundary (or back) of the torso. Two of these landmarks will be positioned closer to the outer boundary of the torso, and the third landmark will be arranged more centrally within the torso. By utilizing these landmarks and the outer boundary of the torso, the fetal overlay can be aligned to divide the image into two sides, for example by extending a line from the outer boundary of the torso closest to the two landmarks, between the two landmarks closest to the outer boundary, and through the third landmark to the other side of the torso.

[0029] As described above, while the fetal overlay divides the fetal image into two sides, additional information may be needed to correctly label the two sides (left and right). Thus, in some aspects, the position or presentation of the fetus in the fetal image (head up, head down, head left, head right, etc.) can be used as this additional information. The presentation of the fetus can be determined in various ways, including but not limited to receiving user input from the sonographer indicating the position of the head, capturing a sagittal view of the fetus by the imaging system in order to detect / determine the presentation, or a combination thereof. In this way, the fetal overlay can indicate the left and right sides of the fetus in the augmented reality image. Additionally, the sonographer can obtain additional images of the fetus when the augmented reality image is displayed, thereby enabling the sonographer to more easily detect and record the position of the fetus and also capture additional images of the fetus's organs when the side of the fetus is being displayed.

[0030] Now referring to Figure 1 , a typical apparatus for an obstetric ultrasound procedure is shown. The apparatus includes an ultrasound machine 100 disposed near a bed 120 or other support structure on which a patient 140 can be located. As Figure 1 shown, the ultrasound machine 100 is typically disposed on the right side of the patient 140, near the patient's head. Such an arrangement provides a standard imaging orientation for the sonographer with respect to the patient 140 and her fetus and allows the sonographer to correctly orient the image of the fetus.

[0031] In Figure 2A functional block diagram of an example ultrasound machine 100 is shown. The ultrasound machine 100 is shown to include a communication device 102, one or more processors 104, a memory 106, a display device 108, and an ultrasound probe 300. The processor 104 may control the operation of the ultrasound machine 100, including implementing at least a portion of the techniques of the present disclosure. The term "processor" as used herein is intended to refer to a single processor and multiple processors that operate together, for example, in a parallel or distributed architecture. The communication device 102 may be configured to communicate with other devices (e.g., various server computing devices, peripheral computing elements, or other computing devices) via a network 200 or other communication connection. A non-limiting example of the communication device 102 is a transceiver, but other forms of hardware are also within the scope of the present disclosure.

[0032] The memory 106 may be any suitable storage medium (e.g., flash memory, hard disk, etc.) configured to store information. For example, the memory 106 may store a set of instructions executable by the processor 104 that cause the ultrasound machine 100 to perform operations (e.g., such as the operations of the present disclosure). The display device 108 may display information to an ultrasound technician and a patient 140. In some implementations, the display device 108 may include a touch-sensitive display device (such as a capacitive touch screen, etc.), but non-touch display devices are also within the scope of the present disclosure. The ultrasound probe 300 is generally a hand-held device that outputs sound waves reflected from various parts of the body. The ultrasound probe 300 also includes a transducer that receives the reflected sound waves and transmits a representation of the sound waves to, for example, the processor 104 for generating an ultrasound image.

[0033] Although the techniques of the present disclosure are described herein in the context of an ultrasound machine 100 and associated components of the ultrasound machine 100, it is specifically contemplated that each feature of the techniques may be performed by a single individual ultrasound machine 100, multiple ultrasound machines 100 operating together, one or more server computing devices, peripheral computing elements, or other computing devices operating in cooperation with one or more ultrasound machines 100, or any combination thereof.

[0034] Further referring to Figure 3 shows an example ultrasound probe 300. The ultrasound probe 300 includes a housing 310 and a connection line 320 that couples the ultrasound probe 300 to other components of the ultrasound machine 100. The ultrasound probe 300 also includes a front portion 330, a left side 340, and a right side 350. The front portion 330 may include a lens or other components suitable for transmitting sound waves from the ultrasound probe 300 and receiving the reflected sound waves. To assist an ultrasound technician in maintaining the correct orientation of the ultrasound probe 300, on one side of the grippable portion 345 of the ultrasound probe, for example, as in Figure 3A protrusion, flange, or other mechanical differentiator (referred to herein as "notch 355") is disposed on the left side 340 shown. In some aspects, and as further described below, the sonographer will align the notch 355 of the ultrasound probe 300 in a specific direction during the ultrasound procedure to maintain an appropriate frame of reference for the fetal image generated by the ultrasound machine 100.

[0035] Further reference is made to Figures 4 to 10 , and techniques for determining the fetal position during a fetal ultrasound procedure using the ultrasound machine 100 are disclosed. In some implementations, for example, a button or icon 410 ( Figure 4 ) on the user interface 400 displayed on the display device 108 can be selected by the sonographer to initiate the fetal position function of the ultrasound machine 100. In response, the ultrasound machine 100 can be operated to determine the fetal position. By way of example only, the ultrasound machine 100 can ask the sonographer to input the fetal position, such as cephalic, breech, transverse, or oblique, as further described below. Additionally or alternatively, the ultrasound machine 100 can use ultrasound scanning to determine the fetal position, such as by prompting the sonographer to capture a specific image using the ultrasound probe 300, and the ultrasound machine 100 can determine the fetal position based on the specific image, for example via image recognition techniques. Other techniques for determining the fetal position are contemplated by the present disclosure.

[0036] As Figure 5A shown, the ultrasound machine 100 can ask the sonographer via the display device 108 of the display user interface 500, which includes selectable icons for various fetal positions. When the fetal position is determined to be cephalic 510 or breech 520, the ultrasound machine 100 can accept the fetal position and proceed with other elements of the technique. However, when the fetal position is transverse 530 or oblique 540, the ultrasound machine 100 may require additional fetal position information. Referring to the transverse fetal position ( Figure 5B ), the ultrasound machine 100 can ask the sonographer to select between the "transverse - head to the right of the pregnant woman" 534 option and the "transverse - head to the left of the pregnant woman" 538 option on the display 108. Referring to the oblique fetal position 540 ( Figure 5C ), the ultrasound machine 100 can ask the sonographer to select between the "oblique - head to the lower right of the pregnant woman" 542 option, the "oblique - head to the upper right of the pregnant woman" 544 option, the "oblique - head to the lower left of the pregnant woman" 546 option, and the "oblique - head to the upper left of the pregnant woman" 548 option on the display 108. Alternatively, referring to Figure 5D , the ultrasound machine 100 can provide graphical elements 550 representing various fetal positions for the sonographer to select.

[0037] After entering the fetal position, now referring to Figure 6 , the ultrasound machine 100 can determine or obtain other information related to the ultrasound procedure being performed. By way of example only, the ultrasound machine 100 can query the sonographer via the display device 108 through the display user interface 600. At 610, the sonographer can be prompted to select or confirm the spatial configuration. In the example shown, the prompt 610 asks the sonographer to confirm that the standard spatial configuration is being used (the ultrasound machine 100 is located on the right side of the pregnant woman). At 620, the ultrasound machine 100 can determine that the ultrasound image being displayed corresponds to the "mid" position, either automatically or requiring confirmation by the sonographer. Finally, at 630, the ultrasound machine 100 can prompt the sonographer to orient the ultrasound probe 300 appropriately. In some aspects, the appropriate orientation is one in which the notch 355 of the ultrasound probe 300 points to the right side of the pregnant woman or towards the head of the patient 140. It should be understood that there are various ways to determine the spatial configuration, image type, and position of the notch 355, all of which are within the scope of the present disclosure. By way of example only, the ultrasound machine 100 and / or the ultrasound probe 300 can include a position / orientation sensor (e.g., an accelerometer or other sensor) that enables the automatic determination and monitoring of the position, orientation, etc. of the notch 355.

[0038] Now referring to Figure 7 , when the ultrasound machine 100 determines or otherwise obtains fetal position information and the spatial configuration, mid-image position, and notch 355 orientation are confirmed, the ultrasound machine 100 can instruct the sonographer to obtain an ultrasound image 700 of the fetus within the uterus. The ultrasound image 700 can be displayed on the display 108 of the ultrasound machine 100. In some implementations, the display 108 can also display a mid-position indicator 705 such that the sonographer can easily confirm that the image is in the mid position during the ultrasound procedure.

[0039] The ultrasound image 700 can include a circumferential view of the torso 710 of the fetus. Additionally, the circumferential view can include the outer boundary 715 of the torso 710 of the fetus and the spine 720. In some aspects, and as mentioned above, the spine 720 will include three landmarks 725-1, 725-2, and 725-3 arranged in a triangular orientation. As described further below, the ultrasound machine 100 and / or the sonographer can utilize the orientation of the outer boundary 715 of the torso and the three landmarks 725-1, 725-2, and 725-3 to determine the fetal position, as described further below.

[0040] Further referring to Figure 8A , Figure 8B and Figure 9, based on the above various inputs, the ultrasound machine 100 can superimpose the fetal overlay 800 on the ultrasound image 700 to obtain an augmented reality image 900 of the fetus. The fetal overlay 800 can include graphical elements 810, 820 indicating the left and right sides of the fetus. The graphical elements 810, 820 can take the form of any visual indicator that differentiates the left side of the fetus from the right side in the augmented reality image 900. By way of example only, in Figure 8A one illustrated example shown, the graphical element includes at least one translucent colored indicator 810-L or 810-R that divides the torso 710 of the fetus into a left (810-L) side and a right (810-R) side. In Figure 8B , the graphical element includes at least one translucent colored indicator 810-L or 810-R that divides the torso 710 of the fetus into a left (810-L) side and a right (810-R) side, as well as a left (820-L) side icon and a right (820-R) side icon (the "L" and "R" are arranged near the appropriate part of the fetus). Other forms of graphical elements are also within the scope of the present disclosure.

[0041] As Figure 8A and Figure 8B shown, the fetal overlay 800 can take two forms: the first form is that when the landmark 725 is arranged towards the bottom of the reference frame ( Figure 8A ), the left side is on the left side of the overlay, and the second form is that when the landmark 725 is arranged towards the bottom of the reference frame ( Figure 8B ), the right side is on the left side of the overlay. When the ultrasound probe 300 is in the correct orientation, which fetal overlay 800 to use is determined based on the above fetal position information (cephalic position 510, breech position 520, transverse position 530 or oblique position 540). By way of example only, Figure 8A 's fetal overlay 800 can be applicable to the cephalic position 510 and the "transverse - head to the right of the pregnant woman" 534 position, while Figure 8B 's fetal overlay 800 can be applicable to the breech position 520 and the "transverse - head to the left of the pregnant woman" 538 position. Additionally, Figure 8A 's fetal overlay 800 can be applicable to the "oblique - head to the lower right of the pregnant woman" 542 position and the "oblique - head to the lower left of the pregnant woman" 546 position, while Figure 8B 's fetal overlay 800 can be applicable to the "oblique - head to the upper right of the pregnant woman" 544 position and the "oblique - head to the upper left of the pregnant woman" 548 position. As described above, the above description depends on the correct orientation of the notch 355 relative to the mother. In some implementations, the ultrasound machine 100 can provide a cue, output, or other reminder to the ultrasound technician to ensure that the notch 355 is correctly oriented when the position of the fetus is input.

[0042] The ultrasound machine 100 may overlay the fetal overlay 800 based on alignment instructions corresponding to the alignment between the outer boundary 715 of the torso 710 and the three landmarks 725 arranged in a triangular orientation. In some implementations, the alignment instructions may be automatically generated by the ultrasound machine 100. By way of example only, the ultrasound machine 100 may use image recognition or other visual matching algorithms on the ultrasound image 700 to detect the outer boundary 715 of the torso 710 and one or more of the three landmarks 725 to generate alignment instructions corresponding to the appropriate position of the fetal overlay 800.

[0043] Alternatively, the ultrasound machine 100 may receive alignment instructions via user input (e.g., from an ultrasound technician) to align the fetal overlay 800 in the proper position. The user input may take various forms, including, but not limited to, input corresponding to positioning and rotating the fetal overlay 800 for aligning the outer boundary 715 of the torso 710 and the landmarks 725 in the fetal overlay 800 with corresponding matching landmarks / icons / elements, etc. In some aspects, a combination of computer-generated and user-entered alignment instructions may be utilized, e.g., where the ultrasound machine 100 generates an initial alignment of the fetal overlay 800 and the ultrasound technician may modify the generated alignment to ensure the desired alignment.

[0044] Now refer to Figure 9 , an example augmented reality image 900 of a fetus is shown as being output by the display device 108. The augmented reality image 900 shown includes a properly aligned outer boundary 715 of the torso 710 and three landmarks 725 arranged in a triangular orientation. Figure 8A The augmented reality image 900 shown also includes a graphical representation 910 of a fetus in a position corresponding to the imaging position of the augmented reality image 900. The graphical representation 910 can depict the orientation of the fetus relative to the augmented reality image 900. In this way, the augmented reality image 900 provides a visual cue to the sonographer regarding the position of the fetus during the current imaging procedure.

[0045] In some aspects, if and when the ultrasound technician changes the position of the augmented reality image 900 (from a neutral position to various rotated views), the graphical representation of the fetus 910 may also change orientation to correspond to how the current image is positioned. Additionally or alternatively, when the ultrasound technician changes the position of the augmented reality image 900 (e.g., by providing instructions to the ultrasound machine 100), the ultrasound machine 100 may adjust the fetal overlay 800 so that the graphical elements 810, 820 correctly indicate the left and right sides of the fetus in the rotated or otherwise altered augmented reality image 900. This may be performed, for example, by the ultrasound machine 100 modifying the fetal overlay 800 in the same manner as the ultrasound image 700 is modified.

[0046] In some implementations in accordance with the present disclosure, the ultrasound machine 100 may automatically adjust the fetal overlay 800 in response to movement of the fetus within the uterus. In such an implementation, the ultrasound machine 100 may detect movement of the fetus in the ultrasound image 700 by utilizing a motion detection algorithm. By way of example only, the motion detection algorithm may store the positions of various markers in the ultrasound image 700 and determine when the various markers have changed position. Based on the detected position changes, the ultrasound machine 100 may determine the position changes (orientation, rotation, amplitude, etc.) of the fetus. Based on the determined position changes, the ultrasound machine 100 may adjust the fetal overlay 800 in a corresponding manner such that the graphical elements 810, 820 correctly indicate the left and right sides of the fetus in the augmented reality image 900. In one example, the ultrasound machine 100 may adjust the fetal overlay based on the alignment between the outer boundary 715 of the torso 710 and three landmarks 725 arranged in a triangular orientation, as described above.

[0047] Now referring to Figure 10 , there is shown an example method 1000 for determining the position of a fetus during an imaging procedure in accordance with some implementations of the present disclosure. For ease of description, method 1000 will be described as being performed by the ultrasound machine 100, however, as described above, method 1000 may also be performed by various different devices working alone or in cooperation. At 1010, the ultrasound machine 100 may obtain an ultrasound image 700 of a fetus within the uterus. In some aspects, the ultrasound image 700 may include (i) a circumferential view of the torso 710 of the fetus and the outer boundary 715 of the torso 710 and (ii) the spine 720 of the fetus. As described above, in certain images, the spine 720 will appear as including three landmarks 725 arranged in a triangular orientation. Based on alignment instructions corresponding to the alignment between the outer boundary 715 of the torso 710 and the three landmarks, the fetal overlay 800 may be superimposed on the ultrasound image 700 to obtain an augmented reality image 900. The augmented reality image 900 and the fetal overlay 800 may include graphical elements 810 and / or 820 indicating the left and right sides of the fetus. The augmented reality image 900 may be output (1030) by the ultrasound device 100.

[0048] In some implementations, method 1000 may further include detecting (1040) fetal movement during the ultrasound procedure. In such an implementation, ultrasound machine 100 may also adjust (1050) the fetal overlay 800 based on the fetal movement such that the graphical elements 810, 820 correctly indicate the left and right sides of the fetus in the augmented reality image 900. Additionally or alternatively, method 1000 may include receiving (1060) an instruction to change the position of the augmented reality image 900. In response to receiving the instruction to change the position 1060, ultrasound machine 100 may automatically adjust the fetal overlay 800 such that the graphical elements 810, 820 correctly indicating the left and right sides of the fetus in the augmented reality image 900 are repositioned.

[0049] In addition to the above, the present technology can be used to perform additional functions regarding the orientation and position of the fetus and its associated body parts. By way of example only, based on the determination of the left and right sides of the fetus as described above, a heart overlay can be generated and superimposed on the heart and / or its various structures. Similar to the fetal overlay described above, the heart overlay may include graphical elements that indicate the anatomical left and right sides of the heart, the various structures / components of the heart (outflow tract, branches, junctions, valves, septum, walls, etc.) or combinations thereof. In an example implementation, the graphical elements may include translucent color indicators that "highlight" and distinguish the different sides of the heart (which may be referred to as "cardiac laterality"), such as presenting the left side of the heart in green and the right side of the heart in red. Other implementations of the graphical elements are also within the scope of the present disclosure.

[0050] In another possible implementation, the present technology can be used to assist in determining the heart position and axis. By way of example only, fetal overlay 800 can be used to automatically generate and / or assist an ultrasound technician in positioning one of two "lines" required to determine the angle between the heart axis and the chest centerline. Once generated / positioned, the ultrasound technician can provide an input indicating the other "line", and the angle can be automatically determined by ultrasound machine 100.

[0051] The present technology can provide further determination based on the fetal overlay 800. As described above, the fetal overlay 800 can be used to display the central axis of the heart ("heart axis"). The ultrasound machine 100 (e.g., in some aspects in combination with the assistance from an ultrasound technician) can utilize the heart axis to perform various calculations and determinations. By way of example only, the technology can include: obtaining the circumferences of the left and right sides of the heart to calculate the ratio between the left and right sides of the heart. Other ratios can include, for example, the ratio of the left circumference to the right circumference / the ratio of the right circumference to the left circumference, the ratio of the left circumference to the total heart circumference, the ratio of the right circumference to the total heart circumference, and the ratio of the heart circumference to the chest circumference. Various other distances and measurements can be calculated, such as the distances between heart landmarks (valves, outflow tracts, etc.) and volume ratios / distance ratios.

[0052] In some implementations, the fetal overlay 800 and associated graphical elements can be extended to not only distinguish different sides of the heart, but also distinguish any associated outflow tracts such as the pulmonary artery, aorta, etc. In this way, the fetal overlay 800 can assist the ultrasound technician in visualizing the outflow tracts, e.g., comparing the shape of such outflow tracts with their expected shape. Additionally, the technology can be used to generate additional direction lines for various outflows to generate angles with other direction lines (e.g., the heart axis). In some embodiments, the fetal overlay 800 can be maintained during a three-dimensional / four-dimensional imaging session to confirm proper positioning and / or determine any unexpected positioning of heart features.

[0053] In some other aspects, the present technology can utilize one landmark or multiple landmarks different from the spine landmark / torso described above to assist in determining the left / right side of the fetus. By way of example only, the arch of the fetus's foot can be used to assist in left / right determination. In a fetal position where the sole of the foot is facing away from the fetal body and the legs, ankles, feet, etc. are anatomically "normal", the arch of the foot can provide an indication of the left and right sides of the fetus. By way of example only, in a cephalic fetal position, the arch of the left foot will appear to have a "C" shape in the mid-image, while the arch of the right foot will appear to have a backward "C" shape in the mid-image (when the heel is down or in the lower / rear part of the ultrasound image). Thus, the fetal overlay 800 can be aligned and utilize the "C" shape of the arch of the foot to align and distinguish the left and right sides of the fetus. Other implementations of landmarks are also within the scope of the present disclosure.

[0054] It should be understood that various techniques of the present disclosure can be combined to confirm the determination of the left / right side of the fetus. By way of example only, spinal landmarks / trunk techniques can be used to determine the left / right side of the fetus, and the fetal foot arch can be used to confirm the determination (alternatively, fetal foot arch techniques can be used to determine the left / right side of the fetus, and spinal landmarks / trunk can be used to confirm the determination). Additionally, the fetal overlay 800 can be extended to display not only the left / right side of the fetus's chest / heart / etc., but also any appendages present during a 3D / 4D imaging session.

[0055] Although the present disclosure has been primarily described in the context of an ultrasound procedure, it should be understood that the techniques described herein are equally applicable (with necessary modifications) to any imaging technique that can be used for fetal imaging. Such techniques can include, but are not necessarily limited to, ultrasound imaging, magnetic resonance imaging (MRI), computed tomography (CT), and X-ray imaging.

[0056] Example embodiments are provided so that this disclosure will be thorough and will fully convey the scope to those skilled in the art. Numerous specific details are set forth such as examples of specific components, devices, and methods to provide a thorough understanding of embodiments of this disclosure. It will be apparent to those skilled in the art that specific details need not be employed, that example embodiments may be embodied in many different forms, and that neither should be construed to limit the scope of this disclosure. In some example embodiments, well-known processes, well-known device structures, and well-known techniques have not been described in detail.

[0057] The terms used herein are for the purpose of describing particular example embodiments only and are not intended to be limiting. Unless the context clearly dictates otherwise, the singular forms "a", "an", and "the" as used herein may also be intended to include the plural forms. The term "and / or" includes any and all combinations of one or more of the associated listed items. The terms "comprises", "comprising", "includes", and "having" are inclusive and thus specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Method steps, processes, and operations described herein should not be construed as necessarily requiring them to be performed in the particular order discussed or illustrated unless specifically identified as an order of performance. It should also be understood that additional steps or alternative steps may be employed.

[0058] Although the terms first, second, third, etc. may be used herein to describe various elements, components, regions, layers, and / or sections, these elements, components, regions, layers, and / or sections should not be limited by these terms. These terms may be used only to distinguish one element, component, region, layer, or section from another. When terms such as "first", "second", and other numerical terms are used herein, they do not imply an order or sequence unless clearly indicated by the context. Thus, the first element, component, region, layer, or section discussed below may be referred to as the second element, component, region, layer, or section without departing from the teachings of the exemplary embodiments.

[0059] As used herein, the term "computing device" or processor may refer to, be part of, or include the following: application specific integrated circuit (ASIC); electronic circuit; combinational logic circuit; field programmable gate array (FPGA); a processor in a networked cluster or data center or a processor in a distributed network (shared, dedicated, or grouped) that executes code or processes and storage; other suitable components that provide the described functionality; or a combination of some or all of the above, such as in a system on a chip. The term "computing device" may also include a memory (shared, dedicated, or grouped) that stores code executed by one or more processors.

[0060] As used above, the term "code" may include software, firmware, bytecode, and / or microcode, and may refer to programs, routines, functions, classes, and / or objects. As used above, the term "shared" means that a single (shared) processor may be used to execute some or all of the code from multiple modules. Additionally, some or all of the code from multiple modules may be stored in a single (shared) memory. As used above, the term "group" means that a group of processors may be used to execute some or all of the code from a single module. Additionally, a group of memories may be used to store some or all of the code from a single module.

[0061] The techniques described herein may be implemented by one or more computer programs executed by one or more processors. The computer programs include processor-executable instructions stored on a non-transitory tangible computer-readable medium. The computer programs may also include the stored data. Non-limiting examples of non-transitory tangible computer-readable media are non-volatile memories, magnetic storage devices, and optical storage devices.

[0062] Some portions of the above description present the techniques described herein in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. While the operations are described functionally or logically, they are to be understood as being implemented by computer programs. Additionally, without loss of generality, it has proven convenient at times to refer to these arrangements of operations as modules or to name them by function.

[0063] Unless explicitly stated otherwise from the foregoing discussion, it should be understood that throughout the specification, discussions using terms such as "processing" or "computing" or "calculating" or "determining" or "displaying" refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system memory or registers or other such information storage, transmission, or display devices.

[0064] Certain aspects of the described techniques include processing steps and instructions described herein in the form of algorithms. It should be noted that the described processing steps and instructions can be embedded in software, firmware, or hardware, and when embedded in software, can be downloaded to reside on and be operated from different platforms used by a real-time network operating system.

[0065] The present disclosure also relates to apparatus for performing the operations herein. The apparatus may be specially constructed for the required purposes or may comprise a general purpose computer selectively activated or reconfigured by a computer program stored on a computer-accessible computer-readable medium. Such a computer program may be stored in a tangible computer-readable storage medium such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical cards, application specific integrated circuits (ASIC), or any type of medium suitable for storing electronic instructions, and each coupled to a computer system bus. Additionally, the computers referred to in the specification may include a single processor or may be an architecture employing multiple processor designs to increase computing capabilities.

[0066] The algorithms and operations presented in this document are not inherently related to any particular computer or other device. A variety of general-purpose systems can also be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized devices to perform the required method steps. For those skilled in the art, the required structure of a variety of such systems, along with equivalent variations, will be apparent. Additionally, the present disclosure has not been described with reference to any particular programming language. It should be understood that various programming languages can be used to implement the teachings of the present disclosure described herein, and any reference to a specific language is provided to disclose the implementation and best mode of the invention.

[0067] The present disclosure is well-suited for a wide variety of computer network systems on many topologies. In this field, the configuration and management of large networks include storage devices and computers that are communicatively coupled to different computers and storage devices via a network such as the Internet.

[0068] For purposes of illustration and description, the foregoing description of the embodiments has been provided. It is not intended to be exhaustive or to limit the present disclosure. The individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but are interchangeable and can be used in the selected embodiment, even if not specifically shown or described. The individual elements or features of a particular embodiment can also vary in many ways. Such variations should not be regarded as a departure from the present disclosure, and all such modifications are intended to be included within the scope of the present disclosure.

Claims

1. A computer-implemented method for determining the position of a fetus during an ultrasound imaging process, comprising: Obtaining an ultrasound image of a fetus in utero at an ultrasound machine having a display and at least one processor, the fetus including a torso and a spine, wherein the ultrasound image includes a circumferential view of the torso of the fetus, and the circumferential view includes at least: (i) the outer boundary of the torso and (ii) the spine including three landmarks arranged in a triangular orientation; At the ultrasound machine, superimposing a fetal overlay based on alignment instructions corresponding to the alignment between the outer boundary of the torso and the three landmarks arranged in the triangular orientation to obtain an augmented reality image of the fetus, the fetal overlay including graphical elements indicating the left and right sides of the fetus; and Outputting the augmented reality image of the fetus on the display of the ultrasound machine.

2. The computer-implemented method according to claim 1, wherein, Receiving, at the ultrasound machine via user input, alignment instructions that align the fetal overlay with the outer boundary of the torso and the three landmarks arranged in the triangular orientation.

3. The computer-implemented method according to claim 1, wherein, Generating, by the ultrasound machine, the alignment instructions based on the alignment between the outer boundary of the torso and the three landmarks arranged in the triangular orientation.

4. The computer-implemented method according to claim 3, further comprising: Detecting, at the ultrasound machine, movement of the fetus in the ultrasound image; And At the ultrasound machine, adjusting the fetal overlay based on the movement of the fetus such that the graphical elements correctly indicate the left and right sides of the fetus in the augmented reality image of the fetus.

5. The computer-implemented method according to claim 4, wherein, Adjusting the fetal overlay based on the movement of the fetus is based on the alignment between the outer boundary of the torso and the three landmarks arranged in the triangular orientation.

6. The computer-implemented method according to claim 1, further comprising: Outputting, on the display of the ultrasound machine, a graphical representation of the fetus depicting the orientation of the fetus relative to the augmented reality image of the fetus.

7. The computer-implemented method according to claim 6, wherein, As the user changes the position of the augmented reality image, the graphical representation of the fetus changes orientation.

8. The computer-implemented method according to claim 1, further comprising: Determining, at the ultrasound machine, an intermediate position of the augmented reality image of the fetus; And When the augmented reality image of the fetus is in the intermediate position, outputting an intermediate position indicator on the display of the ultrasound machine.

9. The computer-implemented method according to claim 1, further comprising: Receiving, at the ultrasound machine, an instruction to change the position of the augmented reality image; And In response to receiving the instruction to change the position, adjusting, at the ultrasound machine, the fetal overlay such that the graphical elements correctly indicate the left and right sides of the fetus in the augmented reality image of the fetus.

10. The computer-implemented method according to claim 1, wherein, The graphical elements indicating the left and right sides of the fetus include at least one translucent color indicator.

11. An ultrasound machine for determining the position of a fetus during an ultrasound imaging process, comprising: A display; One or more processors; And A non-transitory computer-readable storage medium having stored thereon a plurality of instructions which, when executed by the one or more processors, cause the one or more processors to perform operations, the operations including: Obtaining an ultrasound image of a fetus in utero, the fetus including a torso and a spine, wherein the ultrasound image includes a circumferential view of the torso of the fetus, and the circumferential view includes at least: (i) the outer boundary of the torso and (ii) the spine including three landmarks arranged in a triangular orientation; Overlaying a fetal overlay based on alignment instructions corresponding to alignment between the outer boundary of the torso and the three landmarks arranged in the triangular orientation to obtain an augmented reality image of the fetus, the fetal overlay including graphical elements indicating the left and right sides of the fetus; and Outputting the augmented reality image of the fetus on the display.

12. The ultrasonic machine according to claim 11, wherein, Receiving, at the ultrasound machine via user input, alignment instructions that align the fetal overlay with the outer boundary of the torso and the three landmarks arranged in the triangular orientation.

13. The ultrasonic machine according to claim 11, wherein, Generating, by the ultrasound machine, the alignment instructions based on alignment between the outer boundary of the torso and the three landmarks arranged in the triangular orientation.

14. The ultrasonic machine according to claim 13, wherein, The operations further include: Detecting movement of the fetus in the ultrasound image; and Adjusting the fetal overlay based on the movement of the fetus such that the graphical elements correctly indicate the left and right sides of the fetus in the augmented reality image of the fetus.

15. The ultrasonic machine according to claim 14, wherein, Adjusting the fetal overlay based on the movement of the fetus is based on alignment between the outer boundary of the torso and the three landmarks arranged in the triangular orientation.

16. The ultrasonic machine according to claim 11, wherein, The operations further include: Outputting a graphical representation of the fetus on the display, the graphical representation depicting the orientation of the fetus relative to the augmented reality image of the fetus.

17. The ultrasound machine according to claim 16, wherein, As the user changes the position of the augmented reality image, the graphical representation of the fetus changes orientation.

18. The ultrasonic machine according to claim 11, wherein, The operations further include: Determining an intermediate position of the augmented reality image of the fetus; and Outputting an intermediate position indicator on the display when the augmented reality image of the fetus is at the intermediate position.

19. The ultrasonic machine according to claim 11, wherein, The operations further include: Receiving an instruction to change the position of the augmented reality image; and Adjusting the fetal overlay in response to receiving the position change instruction such that the graphical elements correctly indicate the left and right sides of the fetus in the augmented reality image of the fetus.

20. The ultrasonic machine according to claim 11, wherein, The graphical elements indicating the left and right sides of the fetus include at least one translucent color indicator.

Images