Techniques for measuring fetal visceral positions during diagnostic imaging
The method uses augmented reality overlays on ultrasound devices to align fetal landmarks and adjust graphical indicators for accurate fetal visceral position measurement, addressing complexity and inconsistency in current techniques.
Patent Information
- Application Number
- JP2022516207
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-09-11
- Filing Date
- 2020-09-11
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2040-09-11
AI Technical Summary
Current techniques for measuring fetal visceral position during ultrasound diagnosis are complex, confusing, and prone to errors, leading to inconsistencies among sonographers.
A computer-implemented method using ultrasound diagnostic devices with augmented reality overlays to accurately determine fetal visceral position by aligning graphical elements with fetal landmarks, adjusting overlays based on fetal movement, and providing clear indicators for left and right sides.
Enhances the accuracy and simplicity of fetal visceral position measurement, reducing errors and improving consistency among sonographers.
Smart Images

Figure 0007584505000001 
Figure 0007584505000002 
Figure 0007584505000003
Abstract
Description
[Technical field]
[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Application No. 62 / 898,932, filed September 11, 2019. The disclosure of the above application is incorporated herein by reference in its entirety.
[0002] The present disclosure relates generally to improved fetal imaging procedures, and more particularly to techniques for easily measuring and monitoring fetal visceral positions during fetal ultrasound imaging. [Background technology]
[0003] The background description provided herein is intended to provide a general context for the present disclosure. The inventors' work herein to the extent described in this Background section, and aspects of the description that may not otherwise be found to be prior art at the time of filing, are not expressly or impliedly admitted as prior art to the present disclosure.
[0004] Fetal ultrasound is an imaging technique commonly used during pregnancy to monitor fetal development. Among other things, a typical fetal ultrasound checks the fetal internal organs to screen for the presence of abnormalities. For example, a fetal ultrasound may screen the fetus for situs inversus, a condition in which certain internal organs (e.g., heart, stomach, liver) in the fetal chest / abdomen are located on the opposite side of their normal position. In certain circumstances, physicians may wish to provide special care during the delivery or early care of a baby diagnosed with situs in utero. Thus, the ultrasound technician may measure the position of such internal organs during the ultrasound, which may be referred to as "fetal situs."
[0005] Current techniques for determining fetal visceral position during ultrasound examinations are complex, confusing, and cumbersome. Experienced sonographers can utilize a variety of techniques to determine fetal visceral position, and it is not uncommon for two experienced sonographers to disagree about the fetal visceral position of a particular fetus based on such a variety of techniques. Thus, there is a need for improved techniques for determining fetal visceral position during ultrasound examinations. Summary of the Invention
[0006] According to various aspects of the present disclosure, a computer-implemented method for determining fetal visceral positions during ultrasound imaging is disclosed. The method can include acquiring an ultrasound image of a fetus in utero on an ultrasound imaging device having a display and at least one processor. The fetus will include a torso and a spine, and the ultrasound image can include a circumferential view of the torso of the fetus. The circumferential view can include at least (i) an outer border of the torso, and (ii) a spine including three landmarks arranged in a triangular orientation. The method can further include superimposing a fetal overlay on the ultrasound imaging device based on alignment instructions corresponding to an alignment between the outer border of the torso and the three landmarks arranged in a triangular orientation to obtain an augmented reality image of the fetus. The fetal overlay can include graphical elements indicating a left side and a right side of the fetus. The augmented reality image of the fetus can be output on a display of the ultrasound imaging device.
[0007] In some implementations, the alignment instructions can be received at the ultrasound imaging device via a user input that aligns the fetal overlay with the periphery of the torso and the three landmarks arranged in a triangular shape. In additional or alternative implementations, the alignment instructions can be generated at the ultrasound imaging device based on the alignment between the periphery of the torso and the three landmarks arranged in a triangular shape. In such implementations, the method can further include detecting fetal movement in the ultrasound imaging device and adjusting the fetal overlay at the ultrasound imaging device based on the fetal movement such that the graphical element appropriately shows the left and right sides of the fetus in the augmented reality image of the fetus. The adjustment of the fetal overlay based on the fetal movement can be based on the alignment between the periphery of the torso and the three landmarks arranged in a triangular shape.
[0008] In a further aspect, the method can further include outputting a graphical representation of the fetus on a display of the ultrasound imaging device, the graphical representation depicting an orientation of the fetus relative to the augmented reality image of the fetus. The graphical representation of the fetus can change orientation when a user repositions the image.
[0009] In some implementations, the method may further include determining, with the ultrasound diagnostic device, a neutral position of the augmented reality image of the fetus, and outputting a neutral position indicator on a display of the ultrasound diagnostic device when the augmented reality image of the fetus is in a neutral position.
[0010] In some implementations, the method may further include receiving, at the ultrasound imaging device, a command to reposition the augmented reality image, and in response to receiving the command to reposition, adjusting, at the ultrasound imaging device, the fetal overlay such that the graphical elements appropriately indicate the left and right sides of the fetus in the augmented reality image of the fetus. Further, in some implementations, the graphical elements indicating the left and right sides of the fetus may include at least one semi-transparent color indicator.
[0011] In another implementation, the present disclosure is directed to an ultrasound imaging device that includes a display and at least one processor, the ultrasound imaging device capable of performing any, any combination, or all of the methods described above.
[0012] Further areas of applicability of the present disclosure will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. [Brief description of the drawings]
[0013] The present disclosure will become more fully understood from the detailed description and the accompanying drawings, wherein:
[0014] [Figure 1] FIG. 1 is an illustration of a typical positioning of an ultrasound diagnostic device and patient utilized for obstetric ultrasound diagnosis in accordance with some aspects of the present disclosure.
[0015] [Diagram 2] FIG. 1 is a functional block diagram of an exemplary ultrasound diagnostic device in accordance with some aspects of the present disclosure.
[0016] [Diagram 3] 3 is an explanatory diagram of an exemplary ultrasound probe of the exemplary ultrasound diagnostic apparatus of FIG. 2.
[0017] [Figure 4]3 is a view of a display device of the example ultrasound diagnostic device of FIG. 2 when displaying a graphical user interface during an obstetric ultrasound diagnosis, according to some embodiments of the present disclosure.
[0018] [Figure 5A] 3 is a view of a display device of the example ultrasound diagnostic device of FIG. 2 when displaying various graphical user interfaces during an obstetric ultrasound diagnosis in accordance with some embodiments of the present disclosure. [Figure 5B] 3 is a view of a display device of the example ultrasound diagnostic device of FIG. 2 when displaying various graphical user interfaces during an obstetric ultrasound diagnosis in accordance with some embodiments of the present disclosure. [Figure 5C] 3 is a view of a display device of the example ultrasound diagnostic device of FIG. 2 when displaying various graphical user interfaces during an obstetric ultrasound diagnosis in accordance with some embodiments of the present disclosure.
[0019] [Figure 5D] 3 is a view of the display device of the example ultrasound diagnostic apparatus of FIG. 2 when displaying another graphical user interface during an obstetric ultrasound diagnosis, according to some embodiments of the present disclosure.
[0020] [Figure 6] 3 is a view of the display device of the example ultrasound diagnostic device of FIG. 2 when displaying yet another graphical user interface during an obstetric ultrasound diagnosis, in accordance with some embodiments of the present disclosure.
[0021] [Figure 7] 3 is a view of a display device of the example ultrasound diagnostic apparatus of FIG. 2 when displaying an additional graphical user interface during an obstetric ultrasound diagnosis, according to some embodiments of the present disclosure.
[0022] [Figure 8A]1 is an illustration of a fetus overlay superimposed on an ultrasound image to obtain an augmented reality image of the fetus, according to some embodiments of the present disclosure. [Figure 8B] 1 is an illustration of a fetus overlay superimposed on an ultrasound image to obtain an augmented reality image of the fetus, according to some embodiments of the present disclosure.
[0023] [Figure 9] 3 is a view of the display device of the example ultrasound diagnostic apparatus of FIG. 2 when displaying a particular graphical user interface during an obstetric ultrasound diagnosis, in accordance with some embodiments of the present disclosure.
[0024] [Figure 10] FIG. 1 is a flow diagram of an exemplary technique for determining fetal visceral positions during diagnostic imaging, according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0025] A sonographer may take hundreds of images and videos during an ultrasound examination. When imaging a patient directly, the sonographer may have a rough idea of the location of the part of the patient being imaged based on the position of the ultrasound probe relative to the patient. For example, the sonographer may know or easily confirm that he is imaging the right side of the body only if the ultrasound probe is placed on the right side of the patient's body. However, for fetal ultrasound examination, it may not be easy to determine the part of the fetus being imaged because the fetus may be oriented three-dimensionally within the patient's body. Furthermore, the fetus may change position during an ultrasound examination. As will be described more fully herein, it may be important for the sonographer to understand the position and orientation of the fetus within the patient in various aspects of ultrasound examination, including but not limited to measuring fetal laterality and fetal visceral position.
[0026] As briefly described above, current techniques for determining fetal visceral position during ultrasound examinations are complex, confusing, and cumbersome, and errors are common even by experienced sonographers. Accordingly, the present disclosure is directed to improved techniques for determining fetal visceral position. More specifically, the present disclosure describes techniques for determining the left and right sides of a fetus during imaging examinations and generating an augmented reality image of the fetus that is displayed to a sonographer. The augmented reality image includes a fetal overlay superimposed on the fetal image. The fetal overlay can include graphical elements that indicate the left and right sides of the fetus. In an exemplary implementation, the graphical elements may consist of semi-transparent color indicators that "highlight" and distinguish different sides of the fetus, such as presenting the left side of the fetus in green and the right side of the fetus in red. Other implementations of the graphical elements are within the scope of the present disclosure.
[0027] The fetal overlay is aligned with the fetus in the fetal image based on the alignment instructions. The alignment instructions may correspond to an alignment between two or more landmarks of the fetus. In an exemplary implementation, the alignment instructions may be generated using the fetal torso and spine. By way of example only, in a tomographic view of the fetus in a certain position, the fetal image includes a view of the fetal torso periphery and spine. More specifically, as described further below, the spine may appear to have three landmarks, circles, or "points" arranged in a triangular shape and close to the periphery (or back) of the torso. Two of these landmarks will be located closer to the periphery of the torso and the third landmark will be located more centrally of the torso. Using these landmarks and the periphery of the torso, the fetal overlay may be aligned to split the image into two sides, for example, by extending a line from the periphery of the torso closest to the two landmarks, through the two landmarks closest to the periphery and through the third landmark to the opposite side of the torso.
[0028] The fetal overlay divides the fetal image into two sides as described above, but additional information may be needed to correctly label the two sides (left side vs. right side). Thus, in some aspects, the posture or presentation of the fetus in the fetal image (head up, head down, head left, head right, etc.) may be utilized as this additional information. The presentation of the fetus may be measured in a variety of ways, including, but not limited to, receiving user input from the sonographer indicating the head position, capturing a sagittal view of the fetus by the image processing system to detect / measure the presentation, or combinations thereof. In this manner, the fetal overlay may indicate the left and right sides of the fetus in the augmented reality image. Additionally, the sonographer may obtain additional images of the fetus while the augmented reality image is displayed, thereby allowing the sonographer to more easily detect and record the presentation of the fetal internal organs and to capture additional images of the fetal organs while the fetal side is displayed.
[0029] 1, a typical setup utilized for obstetric ultrasound is shown. This setup includes an ultrasound machine 100 positioned adjacent to a bed 120 or other support structure on which a patient 140 may be positioned. The ultrasound machine 100 is typically positioned to the right of the patient 140, adjacent to the patient's head, as shown in FIG. 1. Such a setup provides the ultrasound technician with a standard imaging orientation for the patient 140 and her fetus, allowing the technician to properly orient the image of the fetus.
[0030] A functional block diagram of an example of an ultrasound diagnostic device 100 is shown in FIG. 2. The ultrasound diagnostic device 100 is shown as including a communication device 102, one or more processors 104, a memory 106, a display device 108, and an ultrasound probe 300. The processor(s) 104 can control the operation of the ultrasound diagnostic device 100, including implementing at least a portion of the techniques of the present disclosure. As used herein, the term "processor" is intended to refer to both a single processor and multiple processors operating together, for example, in a parallel or distributed architecture. The communication device 102 can be configured to communicate with another device (e.g., various server computing devices, peripheral computing elements, or other computing devices) via a network 200 or other communication connection. One non-limiting example of the communication device 102 is a transceiver, although other forms of hardware are within the scope of the present disclosure.
[0031] The memory 106 may be any suitable storage medium (e.g., flash, hard disk, etc.) configured to store information. For example, the memory 106 may store a set of instructions executable by the processor 104 to cause the ultrasound diagnostic apparatus 100 to perform operations (e.g., operations of the present disclosure). The display device 108 may display information to the ultrasound technician and the patient 140. In some embodiments, the display device 108 may comprise a touch-sensitive display device (e.g., a capacitive touch screen), although non-touch display devices are also within the scope of the present disclosure. The ultrasound probe 300 is typically a handheld device that outputs sound waves that reflect off body components. The ultrasound probe 300 further includes a transducer that receives the reflected sound waves and transmits a representation of the sound waves, for example to the processor 104, to generate an ultrasound image.
[0032] Although the techniques of the present disclosure are described herein in the context of an ultrasound diagnostic apparatus 100 and associated components of the ultrasound diagnostic apparatus 100, it is expressly contemplated that each feature of the techniques may be performed by only a single ultrasound diagnostic apparatus 100, by multiple ultrasound diagnostic apparatuses 100 operating together, by one or more server computing devices, peripheral computing elements, or other computing devices operating in conjunction with one or more ultrasound diagnostic apparatuses 100, or any combination thereof.
[0033] 3, an exemplary ultrasound probe 300 is illustrated. The ultrasound probe 300 includes a housing 310 and a connecting wire 320 that couples the ultrasound probe 300 to other components of the ultrasound diagnostic device 100. The ultrasound probe 300 further includes a front section 330, a left section 340, and a right section 350. The front section 330 may include a lens or other components suitable for transmitting sound waves from the ultrasound probe 300 and receiving reflected sound waves. To assist the ultrasound technician in maintaining a proper orientation of the ultrasound probe 300, a protrusion, flange, or other mechanical differentiation (referred to herein as a "notch 355") may be located on one lateral side of the graspable portion 345 of the ultrasound probe, for example, on the left section 340 as shown in FIG. 3. In some embodiments, and as further described below, the ultrasound technician will position the notch 355 of the ultrasound probe 300 in a particular orientation during an ultrasound examination to maintain a proper frame of reference for the fetal images generated by the ultrasound diagnostic device 100.
[0034] With further reference to FIGS. 4-10, techniques for measuring fetal visceral position during fetal ultrasound diagnosis using the ultrasound diagnostic device 100 are disclosed. In some embodiments, the fetal visceral position function of the ultrasound diagnostic device 100 may be initiated, for example, by an ultrasound technician selecting a button or icon 410 on a user interface 400 displayed by the display device 108 (see FIG. 4). In response, the ultrasound diagnostic device 100 may operate to measure fetal visceral position. By way of example only, the ultrasound diagnostic device 100 may query the ultrasound technician to input a fetal position, such as cephalic, breech, transverse, or oblique, as described further below. Additionally or alternatively, the ultrasound diagnostic device 100 may measure fetal position using an ultrasound scan, for example, by prompting the ultrasound technician to take a particular image using the ultrasound probe 300, from which the ultrasound diagnostic device 100 can measure fetal position, for example, by image recognition techniques. Other techniques for measuring fetal position are contemplated by the present disclosure.
[0035] As shown in FIG. 5A, the ultrasound diagnostic device 100 can query the ultrasound technician via the display device 108 displaying a user interface 500 that includes selectable icons for various fetal positions. If the fetal position is determined to be cephalic 510 or breech 520, the ultrasound diagnostic device 100 can accept the fetal position and proceed to other elements of the technique. However, if the fetal position is transverse 530 or oblique 540, the ultrasound diagnostic device 100 may request additional fetal position information. With reference to the transverse fetal position (FIG. 5B), the ultrasound diagnostic device 100 can query the ultrasound technician to select from the options of “Transverse-Head to Mother’s Right” 534 or “Transverse-Head to Mother’s Left” 538 on the display 108. With reference to oblique fetal position 540 (FIG. 5C), the ultrasound diagnostic device 100 can prompt the sonographer to select from the following options on the display 108: "oblique-head facing down to the right of mother" 542, "oblique-head facing up to the right of mother" 544, "oblique-head facing down to the left of mother" 546, and "oblique-head facing up to the left of mother" 548. Alternatively, and with reference to FIG. 5D, the ultrasound diagnostic device 100 may provide a graphical element 550 displaying various fetal positions for selection by the sonographer.
[0036] Upon input of the fetal position, and now referring to FIG. 6 , the ultrasound diagnostic device 100 can measure or obtain further information related to the ultrasound diagnosis being performed. By way of example only, the ultrasound diagnostic device 100 can query the ultrasound technician via the display device 108 by displaying a user interface 600. At 610, the ultrasound technician can be prompted to select or confirm a room configuration. In the illustrated example, the prompt 610 queries the ultrasound technician to confirm that a standard room configuration (ultrasound device 100 positioned to the maternal right side) is being utilized. At 620, the ultrasound diagnostic device 100 can determine, e.g., automatically or by requesting confirmation by the ultrasound technician, that the displayed ultrasound image corresponds to a “neutral” position. Finally, at 630, the ultrasound diagnostic device 100 can prompt the ultrasound technician to position the ultrasound probe 300 in a proper orientation. In some embodiments, the proper orientation is with the notch 355 of the ultrasound probe 300 positioned to the maternal right side or toward the head of the patient 140. It should be appreciated that there are a variety of ways in which the room configuration, image type, and location of the notch 355 may be determined, all of which are within the scope of this disclosure. By way of example only, the ultrasound diagnostic device 100 and / or ultrasound probe 300 may include a position / orientation sensor (e.g., an accelerometer or other sensor) that allows for automatic determination and monitoring of the location, orientation, etc. of the notch 355.
[0037] 7, once ultrasound diagnostic device 100 has measured or otherwise acquired fetal position information and the room configuration, neutral image position, and orientation of notch 355 have been confirmed, ultrasound diagnostic device 100 can instruct the sonographer to acquire an ultrasound image 700 of the fetus in utero. Ultrasound image 700 can be displayed on display 108 of ultrasound diagnostic device 100. In some implementations, display 108 can also display a neutral position indicator 705 to allow the sonographer to easily verify that the image is in a neutral position during the ultrasound examination.
[0038] The ultrasound image 700 may include a cross-sectional view of the fetal torso 710. Additionally, the cross-sectional view may include an outer edge 715 of the torso 710 and the fetal spine 720. In some aspects, and as described above, the spine 720 may include three landmarks 725-1, 725-2, and 725-3 arranged in a triangular shape. As described further below, the ultrasound diagnostic device 100 and / or the ultrasound technician may use the orientation of the outer edge 715 of the torso and the three landmarks 725-1, 725-2, and 725-3 to determine the position of the fetal internal organs.
[0039] 8A, 8B, and 9, based on the various inputs described above, the ultrasound diagnostic device 100 can overlay a fetal overlay 800 onto the ultrasound image 700 to obtain an augmented reality image 900 of the fetus. The fetal overlay 800 can include graphical elements 810, 820 indicating the left and right sides of the fetus. The graphical elements 810, 820 can take the form of any visual indicator that distinguishes the left side of the fetus from the right side of the fetus in the augmented reality image 900. By way of example only, in the illustrative example shown in FIG. 8A, the graphical elements include at least one semi-transparent color indicator 810-L or 810-R that bisects the fetal torso 710 into a left side (810-L) and a right side (810-R). In Figure 8B, the graphical element includes at least one semi-transparent color indicator 810-L or 810-R that bisects the fetal torso 710 into left (810-L) and right (810-R) sides, and left (820-L) and right (820-R) icons (with the "L" and "R" positioned adjacent to the appropriate portions of the fetus). Other forms of the graphical element are within the scope of this disclosure.
[0040] As shown in Figures 8A and 8B, the fetal overlay 800 can take two forms, a first form with the left side on the left side of the overlay with the landmarks 725 located towards the bottom of the frame (Figure 8A) and a second form with the right side on the left side of the overlay with the landmarks 725 located towards the bottom of the frame (Figure 8B). The decision of which fetal overlay 800 to utilize is made based on the fetal position information described above (capitate 510, breech 520, transverse 530, or oblique 540) when the ultrasound probe 300 is in the appropriate orientation. By way of example only, the fetal overlay 800 of Figure 8A may be appropriate for a capitate 510 and a "transverse-head to maternal right" 534 position, and the fetal overlay 800 of Figure 8B may be appropriate for a breech 520 and a "transverse-head to maternal left" 538 position. Additionally, the fetal overlay 800 of FIG. 8A may be appropriate for the "oblique-head below mother to the right" 542 and "oblique-head below mother to the left" 546 positions, and the fetal overlay 800 of FIG. 8B may be appropriate for the "oblique-head above mother to the right" 544 and "oblique-head above mother to the left" 548 positions. The above description relies on the proper orientation of the notch 355 relative to the mother, as discussed above. In some embodiments, the ultrasound diagnostic device 100 may provide a prompt, output, or other reminder to the ultrasound technician to properly orient the notch 355 when the fetal position is entered.
[0041] The ultrasound diagnostic device 100 can superimpose the fetal overlay 800 based on alignment instructions corresponding to an alignment between the outer edge 715 of the torso 710 and the three landmarks 725 arranged in a triangular shape. In some implementations, the alignment instructions can be generated automatically by the ultrasound diagnostic device 100. By way of example only, the ultrasound diagnostic device 100 can generate alignment instructions corresponding to the appropriate position of the fetal overlay 800 by detecting the outer edge 715 of the torso 710 and one or more of the three landmarks 725 using image recognition or other visual matching algorithms on the ultrasound image 700.
[0042] Alternatively, the ultrasound diagnostic device 100 may receive alignment instructions via user input (e.g., from a sonographer) that aligns the fetal overlay 800 in the appropriate position. User input can take a variety of forms, including, but not limited to, input corresponding to the position and rotation of the fetal overlay 800 to align the outer edge 715 and landmarks 725 of the torso 710 with corresponding matching landmarks / icons / elements / etc. in the fetal overlay 800. In some aspects, a combination of computer-generated and user-entered alignment instructions may be utilized, for example, the ultrasound diagnostic device 100 may generate an initial alignment of the fetal overlay 800 and allow the sonographer to modify the generated alignment to ensure the desired alignment.
[0043] 9, an exemplary augmented reality image 900 of a fetus is shown as output by the display device 108. The illustrated augmented reality image 900 includes the fetus overlay 800 of FIG. 8A in proper alignment between the outer edge 715 of the torso 710 and three landmarks 725 arranged in a triangular shape. The illustrated augmented reality image 900 further includes a graphical representation 910 of the fetus in a position corresponding to the imaging position of the augmented reality image 900. The graphical representation 910 may depict the orientation of the fetus relative to the augmented reality image 900. In this aspect, the augmented reality image 900 provides a visual cue to the sonographer regarding the position of the fetus during the current imaging procedure.
[0044] In some aspects, if and when the sonographer changes the position of the augmented reality image 900 (from a neutral position to various rotated views), the graphical representation of the fetus 910 may also change orientation depending on how the current image is positioned. Additionally or alternatively, when the sonographer changes the position of the augmented reality image 900 (e.g., by providing instructions to the ultrasound diagnostic device 100), the ultrasound diagnostic device 100 may adjust the fetal overlay 800 so that the graphical elements 810, 820 properly show the left and right sides of the fetus in the rotated or otherwise altered augmented reality image 900. This may be done, for example, by the ultrasound diagnostic device 100 altering the fetal overlay 800 in the same manner that the ultrasound image 700 is altered.
[0045] According to some implementations of the present disclosure, the ultrasound diagnostic device 100 may automatically adjust the fetal overlay 800 in response to fetal movement within the uterus. In such implementations, the ultrasound diagnostic device 100 may detect fetal movement in the ultrasound image 700 by utilizing a motion detection algorithm. By way of example only, the motion detection algorithm may store the positions of various markers within the ultrasound image 700 and determine when the various markers change position. Based on the detected position change, the ultrasound diagnostic device 100 may determine the fetal pose change (e.g., orientation, rotation, magnitude of movement, etc.). Based on the determined pose change, the ultrasound diagnostic device 100 may adjust the fetal overlay 800 in a corresponding manner such that the graphical element(s) 810, 820 properly indicate the left and right sides of the fetus within the augmented reality image 900. In one example, the ultrasound diagnostic device 100 may adjust the fetal overlay based on the alignment between the outer edge 715 of the torso 710 and the three landmarks 725 arranged in a triangular shape, as described above.
[0046] 10, an exemplary method 1000 for determining fetal visceral positions during diagnostic imaging according to some embodiments of the present disclosure is illustrated. Although the method 1000 may be performed by a variety of different devices, either alone or in concert, as described above, for ease of explanation, the method 1000 is described as being performed by an ultrasound imaging device 100. At 1010, the ultrasound imaging device 100 may acquire an ultrasound image 700 of a fetus in utero. In some embodiments, the ultrasound image 700 may include a cross-sectional view of a fetal torso 710 and an outer edge 715 of the torso 710, and (ii) a fetal spine 720. As described above, in certain images, the spine 720 may appear to include three landmarks 725 arranged in a triangular shape. Based on alignment instructions corresponding to the alignment of the outer edge 715 of the torso 710 with the three landmarks, a fetal overlay 800 may be superimposed on the ultrasound image 700 to obtain an augmented reality image 900. The augmented reality image 900 and fetus overlay 800 may include graphical elements 810 and / or 820 indicating the left and right sides of the fetus. The augmented reality image 900 may be output (1030) by the ultrasound diagnostic device 100.
[0047] In some implementations, the method 1000 may further include detecting 1040 fetal movement during the ultrasound examination. In such implementations, the ultrasound diagnostic device 100 may also adjust 1050 the fetal overlay 800 based on the fetal movement such that the graphical elements 810, 820 properly indicate the left and right sides of the fetus in the augmented reality image 900. Additionally or alternatively, the method 1000 may include receiving 1060 an instruction to change the position of the augmented reality image 900. In response to receiving 1060 the position change instruction, the ultrasound diagnostic device 100 may automatically adjust 1040 the fetal overlay 800 such that the graphical elements 810, 820 properly indicate the left and right sides of the fetus in the repositioned augmented reality image 900.
[0048] In addition to the above, the present techniques may be utilized to perform additional functions with respect to the orientation and position of the fetus and its associated body parts. By way of example only, based on the described determination of the left and right side of the fetus, a cardiac overlay may be generated and superimposed over the heart and / or its various structures. Similar to the fetal overlay described above, the cardiac overlay may include graphical elements showing the anatomical left and right sides of the heart, its various structures / components (outflow tracts, branches, confluences, valves, septa, walls, etc.), or combinations thereof. In an exemplary embodiment, the graphical elements may include semi-transparent color indicators that "highlight" and distinguish different sides of the heart (which may be referred to as "cardiac laterality"), such as presenting the left side of the heart in green and the right side of the heart in red. Other implementations of the graphical elements are within the scope of this disclosure.
[0049] In another possible embodiment, the technique can be utilized to assist in determining the position and axis of the heart. By way of example only, fetal overlay 800 can be utilized to automatically generate and / or assist the sonographer in positioning one of the two "lines" required to measure the angle between the axis of the heart and the thoracic centerline. Once generated / positioned, the sonographer can provide input indicating the other "line" and the angle can be automatically measured by ultrasound diagnostic device 100.
[0050] The technique can provide further determinations based on the fetal overlay 800. As described above, the fetal overlay 800 can be utilized to display the central axis of the heart ("heart axis"). The ultrasound diagnostic device 100 (e.g., in conjunction with assistance from an ultrasound technician in some embodiments) can utilize the heart axis to perform various calculations and determinations. By way of example only, the technique can include obtaining the circumferences of the left and right sides of the heart to calculate the ratio between the left and right sides of the heart. Other ratios can include, for example, left side circumference to right side circumference / right side circumference to left side circumference, left side circumference to total heart circumference, right side circumference to total heart circumference, and heart circumference to rib cage circumference. Various other distances and measurements can be calculated, such as distances between cardiac landmarks (valves, outflow tract, etc.) and volume / distance ratios.
[0051] In some embodiments, the fetal overlay 800 and associated graphical elements can be extended to not only distinguish between different sides of the heart, but also any associated outflow tracts, such as the pulmonary artery, aorta, etc. In this manner, the fetal overlay 800 can assist the sonographer in visualizing such outflow tracts, for example, to compare the shape of such tracts to their expected shape. Additionally, this technique may be utilized to generate additional direction lines of the various outflow tracts to generate angles with other direction lines (such as the heart axis). In some embodiments, the fetal overlay 800 can remain in a 3 or 4 dimensional imaging session to confirm proper positioning and / or determine unexpected positioning of cardiac features.
[0052] In some further aspects, the technology can utilize one or more landmarks, different from the spinal landmarks / torso described above, to assist in determining the left / right side of the fetus. By way of example only, the arch of the fetal foot can be utilized to assist in the determination of the left / right side. In a fetal position where the soles of the fetal feet are pointing away from the fetal body and the legs, ankles, feet, etc. are anatomically "normal", the arch of the foot can provide an indication of the left and right side of the fetus. By way of example only, in a head-down fetal position, in a neutral image (when the heels are down or in the lower / back portion of the ultrasound image), the arch of the left foot will appear to have a "C" shape and the right foot will appear to have an inverted "C" shape. Thus, the fetal overlay 800 can utilize the "C" shape of the arch to register and differentiate between the left and right sides of the fetus. Other implementations of landmarks are within the scope of this disclosure.
[0053] It should be understood that various techniques of the present disclosure may be utilized in combination to confirm the fetal left / right side determination. By way of example only, spinal landmark / torso techniques may be utilized to determine the fetal left / right side and fetal foot arches may be utilized to confirm this determination (or vice versa). Additionally, fetal overlay 800 may be expanded to not only display the left / right side of the fetal rib cage / heart / etc., but also any appendages present during a 3 or 4 dimensional imaging session.
[0054] Although the present disclosure is described primarily in the context of ultrasound imaging, it should be understood that the techniques described herein are equally applicable, mutatis mutandis, to any imaging technique available for imaging the fetus, which may include, but are not necessarily limited to, ultrasound imaging, magnetic resonance imaging (MRI), computed tomography (CT) scans, and x-ray imaging.
[0055] Exemplary embodiments are provided so that this disclosure will be thorough and fully convey the scope to those skilled in the art. Numerous specific details are described, such as examples of specific components, devices, and methods, to provide a thorough understanding of the embodiments of the present disclosure. It will be apparent to those skilled in the art that specific details need not be employed, that the exemplary embodiments can be embodied in many different forms, and neither should be construed to limit the scope of the present disclosure. In some exemplary embodiments, well-known procedures, well-known device structures, and well-known technologies are not described in detail.
[0056] The terms used herein are for the purpose of describing particular exemplary embodiments only and are not intended to be limiting. As used herein, the singular forms "a," "an," and "the" may be intended to include the plural forms unless the context clearly indicates otherwise. The term "and / or" includes any and all combinations of one or more of the associated listed items. The terms "comprises," "comprising," "including," and "have" are inclusive and thus specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. The method steps, processes, and operations described herein should not be construed as necessarily requiring their execution in the particular order discussed or illustrated, unless specifically identified as an order of execution. It is also understood that additional or alternative steps may be employed.
[0057] In this specification, terms such as first, second, third, etc. may be used to describe various elements, components, regions, layers, and / or sections, but these elements, components, regions, layers, and / or sections should not be limited by these terms. These terms may be used only to distinguish one element, component, region, layer, or section from another region, layer, or section. As used herein, terms such as "first," "second," and other numerical terms do not imply any order or sequence unless clearly indicated by the context. Thus, a first element, component, region, layer, or section described below can be referred to as a second element, component, region, layer, or section without departing from the teachings of the exemplary embodiments.
[0058] As used herein, the term "computing device" or processor may refer to, be a part of, or include: an application specific integrated circuit (ASIC); an electronic circuit; a combinatorial logic circuit; a field programmable gate array (FPGA); a processor or a distributed network of processors (shared, dedicated, or grouped) and storage in a network cluster or data center that executes code or processes; other suitable components that provide the described functionality; or a combination of some or all of the above, such as a system on a chip. The term computing device may also include memory (shared, dedicated, or grouped) that stores code executed by one or more processors.
[0059] As used above, the term code includes software, firmware, bytecode, and / or microcode and may refer to programs, routines, functions, classes, and / or objects. As used above, the term shared means that some or all code from multiple modules may be executed using a single (shared) processor. Furthermore, some or all code from multiple modules may be stored by a single (shared) memory. As used above, the term group means that some or all code from a single module may be executed using a group of processors. Furthermore, some or all code from a single module may be stored using a group of memories.
[0060] The techniques described herein may be implemented by one or more computer programs executed by one or more processors. The computer programs include processor-executable instructions stored on a non-transitory, tangible computer-readable medium. The computer programs may also include stored data. Non-limiting examples of non-transitory, tangible computer-readable media are non-volatile memory, magnetic storage devices, and optical storage devices.
[0061] Some portions of the above description are presented in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. These operations, while described in functional or logical terms, are understood to be performed by computer programs. Further, it has proven convenient at times to refer to the arrangement of these operations as modules or by functional names, without loss of generality.
[0062] As is evident from the above discussion, unless specifically stated otherwise, discussions throughout this specification utilizing terms such as "processing" or "computing" or "calculating" or "determining" or "displaying" will be understood to refer to the actions and processing of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within the memory or registers of a computer system or other such information storage, transmission, or display device.
[0063] Certain aspects of the described technology include process steps and instructions described herein in the form of an algorithm. It should be noted that the described process steps and instructions may be embodied in software, firmware, or hardware, and if embodied in software, may be downloaded to reside on and be operated from different platforms used by a real-time network operating system.
[0064] The present disclosure also relates to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in a computer-readable medium accessible by the computer. Such a computer program may be stored in a tangible computer-readable storage medium, such as, but not limited to, a floppy disk, an optical disk, a CD-ROM, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic or optical card, an application specific integrated circuit (ASIC), or any type of medium suitable for storing electronic instructions, each of which is coupled to a computer system bus. Furthermore, the computers referred to herein may include a single processor or may be architectures employing multiple processor designs to increase computing power.
[0065] The algorithms and operations presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems, along with equivalent variations, will be apparent to those skilled in the art. Additionally, this disclosure is not described with reference to any particular programming language. It will be understood that a variety of programming languages can be used to implement the teachings of the disclosure described herein, and any reference to a specific language is provided for the purpose of disclosing the enablement and best mode of the invention.
[0066] The present disclosure is well suited to a wide range of computer network systems across multiple topologies, in the field of configuration and management of large scale networks including storage devices and computers communicatively coupled to disparate computers and storage devices over networks such as the Internet.
[0067] The foregoing description of the embodiments has been provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the present disclosure. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but even if not specifically shown or described, can be interchangeable and used in selected embodiments, where applicable. The same can also be varied in many ways. Such variations are not to be considered as departures from the present disclosure, and all such modifications are intended to be included within the scope of the present disclosure. [Explanation of symbols]
[0068] 100: ultrasound diagnostic device, 104: processor, 108: display device, 700: ultrasound image, 705: neutral position indicator, 710: torso, 715: periphery, 720: spine, 725 (725-1, 725-2, 725-3): landmarks, 800: fetal overlay, 810, 820: graphical elements, 810-L, 810-R: semi-transparent color indicators, 900: augmented reality image, 910: graphical representation
Claims
1. 1. A computer-implemented method for determining fetal visceral positions during ultrasound imaging, comprising: acquiring an ultrasound image of a fetus in utero in an ultrasound diagnostic device having a display and at least one processor, the fetus including a torso and a spine, the ultrasound image including a cross-sectional view of the torso of the fetus, the cross-sectional view including at least (i) an outer edge of the torso, and (ii) the spine including three landmarks arranged in a triangular shape; In the ultrasound diagnostic device, to obtain an augmented reality image of the fetus, a fetal overlay indicating information about the fetus is superimposed on the ultrasound image based on an alignment command corresponding to alignment between the outer edge of the torso and the three landmarks arranged in the triangular shape, the fetal overlay including graphical elements indicating the left and right sides of the fetus; and outputting the augmented reality image of the fetus on the display of the ultrasound diagnostic device; the alignment command is generated by the ultrasound diagnostic device based on the registration between the outer edge of the torso and the three landmarks arranged in the triangular shape; detecting fetal movement in the ultrasound image in the ultrasound diagnostic device; and and adjusting the fetal overlay in the ultrasound imaging device based on the movement of the fetus so that the graphical element appropriately represents the left and right sides of the fetus in the augmented reality image of the fetus.
2. 2. The computer-implemented method of claim 1, wherein the adjustment of the fetal overlay based on the movement of the fetus is based on the alignment between the outer edge of the torso and the three landmarks arranged in the triangular shape.
3. 10. The computer-implemented method of claim 1, further comprising outputting a graphical representation of the fetus on the display of the ultrasound diagnostic device, the graphical representation depicting an orientation of the fetus relative to the augmented reality image of the fetus.
4. 4. The computer-implemented method of claim 3, wherein the graphical representation of the fetus changes orientation as a user changes position of the augmented reality image.
5. determining a neutral position of the augmented reality image of the fetus in the ultrasound diagnostic device; and 10. The computer-implemented method of claim 1, further comprising outputting a neutral position indicator on the display of the ultrasound diagnostic device when the augmented reality image of the fetus is in the neutral position.
6. receiving, at the ultrasound diagnostic device, a command to change the position of the augmented reality image; and 2. The computer-implemented method of claim 1, further comprising, in response to receiving the command to change position, adjusting the fetal overlay on the ultrasound imaging device so that the graphical element appropriately represents the left and right sides of the fetus in the augmented reality image of the fetus.
7. 10. The computer-implemented method of claim 1, wherein the graphical elements representing the left and right sides of the fetus include at least one semi-transparent color indicator.
8. 1. An ultrasound diagnostic device for determining fetal visceral positions during ultrasound imaging, comprising: The display and one or more processors; a non-transitory computer-readable storage medium having stored therein a plurality of instructions that, when executed by the one or more processors, cause the one or more processors to perform operations; The operation is obtaining an ultrasound image of a fetus in utero, the fetus including a torso and a spine, the ultrasound image including a cross-sectional view of the torso of the fetus, the cross-sectional view including at least (i) a periphery of the torso, and (ii) the spine including three landmarks arranged in a triangular configuration; superimposing a fetal overlay showing information about the fetus on the ultrasound image based on alignment instructions corresponding to alignment between the outer edge of the torso and the three landmarks arranged in the triangular shape to obtain an augmented reality image of the fetus, wherein the fetal overlay includes graphical elements showing the left and right sides of the fetus; and outputting the augmented reality image of the fetus on the display; the alignment command is generated by the ultrasound diagnostic device based on the registration between the outer edge of the torso and the three landmarks arranged in the triangular shape; The operation is detecting fetal movement in the ultrasound image; and adjusting the fetal overlay based on the movement of the fetus so that the graphical element appropriately represents the left and right sides of the fetus in the augmented reality image of the fetus.
9. 9. The ultrasound diagnostic device of claim 8, wherein the adjustment of the fetal overlay based on the movement of the fetus is based on the alignment between the outer edge of the torso and the three landmarks arranged in the triangular shape.
10. 10. The ultrasound diagnostic device of claim 8, wherein the operations further include outputting a graphical representation of the fetus on the display, the graphical representation depicting an orientation of the fetus relative to the augmented reality image of the fetus.
11. 11. The ultrasound diagnostic device of claim 10, wherein the graphical representation of the fetus changes orientation when a user changes the position of the augmented reality image.
12. The operation is determining a neutral position of the augmented reality image of the fetus; and 9. The ultrasound diagnostic device of claim 8, further comprising outputting a neutral position indicator on the display when the augmented reality image of the fetus is in the neutral position.
13. The operation is receiving an instruction to change the position of the augmented reality image; and 10. The ultrasound diagnostic device of claim 8, further comprising: in response to receiving the command to change position, adjusting the fetal overlay so that the graphical element appropriately represents the left and right sides of the fetus in the augmented reality image of the fetus.
14. 9. The ultrasound diagnostic device of claim 8, wherein the graphical elements representing the left and right sides of the fetus include at least one semi-transparent color indicator.
Citation Information
Patent Citations
Ultrasonograph
JP1995116159A
Ultrasonic diagnosis device, ultrasonic image processing apparatus, ultrasonic image acquisition program, and ultrasonic diagnosis display program
JP2010115483A
Ultrasonic diagnostic apparatus
JP2010187987A
Ultrasonic diagnostic apparatus and image processing apparatus
JP2013074941A
Ultrasonic diagnostic device
JP2014008083A
