CIRCUIT DESIGN AND PROCEDURE

Abstract

The disclosure relates to a circuit arrangement for generating a visualization of a fetus, wherein the circuit arrangement is configured to: receive ultrasound data representing a fetus in a uterus of its mother; create, based on the ultrasound data, a data model of the fetus; and generate, based on the data model, a visualization of the fetus using generative artificial intelligence.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to a circuit arrangement and a method, in particular to a circuit arrangement and a method for generating a visualization of a fetus. TECHNICAL BACKGROUND

[0002] Ultrasound imaging of a fetus is well known. Ultrasound imaging is sometimes based on emitting an ultrasound signal into a mother's abdomen while the fetus is in the mother's uterus and detecting a reflection of the ultrasound signal. Based on the intensity of the reflection and the delay between the emission of the ultrasound signal and the detection of the reflection, an image of the fetus is reconstructed.

[0003] Although techniques for ultrasound imaging of a fetus exist, it is generally desirable to provide improved circuitry and method for producing visualization of a fetus. SUMMARY

[0004] According to a first aspect, the disclosure provides circuitry for generating a visualization of a fetus, the circuitry being configured to: obtain ultrasound data representing a fetus in a uterus of its mother; create, based on the ultrasound data, a data model of the fetus; and generate, based on the data model, a visualization of the fetus using generative artificial intelligence.

[0005] According to a second aspect, the disclosure provides a method for generating a visualization of a fetus, the method comprising: obtaining ultrasound data representing a fetus in a uterus of its mother; creating, based on the ultrasound data, a data model of the fetus; and generating, based on the data model, a visualization of the fetus using generative artificial intelligence.

[0006] Further aspects are set out in the dependent claims, the drawings and the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Embodiments are explained by way of example with reference to the accompanying drawings, in which: Fig. 1 illustrates an embodiment of a circuit arrangement; Fig. 2 illustrates an embodiment of a method; and Fig. 3 illustrates an embodiment of a general-purpose computer. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0008] Before a detailed description of the embodiments with reference to Fig. 1, general explanations are given.

[0009] As mentioned at the beginning, it is well known to perform ultrasound imaging of a fetus. Ultrasound imaging can be based on emitting an ultrasound signal into a mother's abdomen while the fetus is in the mother's uterus and detecting a reflection of the ultrasound signal. Based on the intensity of the reflection as well as the delay between the emission of the ultrasound signal and the detection of the reflection, an image of the fetus can be reconstructed.

[0010] For example, a wonderful process for expectant parents during pregnancy can be to undergo an ultrasound during a visit to a doctor (typically a gynecologist) and view a real-time image of the baby inside the womb. In order for the doctor to obtain a good scan, it can also be very important to take precise measurements and understand the baby's size, position, and certain aspects of its growth to ensure safe development or to initiate certain countermeasures and treatments as early as possible if any problems are noticed.

[0011] However, in some cases, ultrasound examination typically provides a two-dimensional (2D) image, and good skills in positioning an ultrasound probe and reading the image are required to obtain a correct image and an understanding of what exactly is visible in the image.

[0012] For a layperson, such as typical expectant parents, it can be extremely difficult to understand different body parts, and especially the position and size of the baby within the womb, based on the image. Although some three-dimensional (3D) reconstruction algorithms may already exist in medical ultrasound devices, such existing 3D reconstruction algorithms may still not provide a clear image and understanding of the baby's condition and posture (e.g., position, orientation).

[0013] Accordingly, some embodiments of the disclosure relate to circuitry for generating a visualization of a fetus, the circuitry being configured to: obtain ultrasound data representing a fetus in a uterus of its mother; create, based on the ultrasound data, a data model of the fetus; and generate, based on the data model, a visualization of the fetus using generative artificial intelligence.

[0014] The circuitry may be configured to execute instructions (e.g., software, firmware) and may include a programmed microprocessor (e.g., a central processing unit, CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or the like. The instructions may cause the circuitry to provide functionality related to generating a visualization of a fetus.

[0015] The circuitry may include a memory that stores the instructions, the obtained ultrasound data, the data model of the fetus, the generated visualization, and / or (other) temporary values ​​generated by the circuitry. The memory may include flash memory, electrically erasable programmable read-only memory (EEPROM), dynamic random access memory (DRAM), static random access memory (SRAM), or the like.

[0016] The circuitry may include a communication unit for receiving the ultrasound data and / or outputting the generated visualization of the fetus. The communication unit may be configured to communicate with another device via Universal Serial Bus (USB), Peripheral Component Interconnect (PCI), serial port (RS-232), parallel port (IEEE 1284), Ethernet, wireless local area network (WLAN; e.g., Wi-Fi, IEEE 802.11), Bluetooth, High-Definition Multimedia Interface (HDMI), DisplayPort, or the like.

[0017] The circuit arrangement may include an artificial intelligence processing unit for executing the generative artificial intelligence. The artificial intelligence processing unit may include a graphics processing unit (GPU) and / or a tensor processing unit (TPU).

[0018] The circuit arrangement can be implemented, for example, as a general-purpose computer 150, as described with reference to Fig. 3 described.

[0019] The circuit arrangement may receive the ultrasound data from an ultrasound probe. The ultrasound probe may include an ultrasound source and an ultrasound sensor. The ultrasound source may be configured to emit a predefined ultrasound signal. The ultrasound signal may have a frequency in a range between 1 MHz and 40 MHz, without limiting the disclosure to this range. The ultrasound sensor may be configured to detect a reflection of the ultrasound signal, including an intensity of the reflection and a delay between the emission of the ultrasound signal by the ultrasound source and the detection of the corresponding reflection by the ultrasound sensor. The ultrasound source and / or the ultrasound sensor may include a piezoelectric element for generating the ultrasound signal and / or for detecting the reflection, each based on the piezoelectric effect.Other technologies for generating the ultrasonic signal and / or detecting the reflection may also be possible.

[0020] When a user (e.g., a physician performing a fetal ultrasound scan) positions the ultrasound probe against the mother's abdomen, the ultrasound source can emit the ultrasound signal into the mother's abdomen, allowing the ultrasound signal to reach the mother's uterus and the fetus within the uterus. Different materials and / or tissue types within the abdomen, uterus, and / or fetus can absorb the ultrasound signal with different specific absorption coefficients and can attenuate the ultrasound signal accordingly. Inhomogeneity (e.g., an interface between different materials and / or tissue types) within the abdomen, uterus, or fetus (e.g., a surface of the fetus) can reflect the ultrasound signal.The ultrasonic sensor can detect the reflection including its intensity and / or its delay after an emission of the respective ultrasonic signal.

[0021] The ultrasound source may emit multiple ultrasound pulses in the ultrasound signal and / or may emit multiple ultrasound signals, and the ultrasound sensor may detect the respective multiple ultrasound pulses and / or ultrasound signals. For example, a user may move the ultrasound probe to different positions on the mother's abdomen to obtain ultrasound data of the fetus from different perspectives and / or to find an optimal position from which to image a part of interest of the fetus. For example, the ultrasound source may emit subsequent ultrasound pulses in different directions and / or the ultrasound sensor may determine intensities and delays of respective reflections of ultrasound pulses from different directions, thereby increasing a field of view of the ultrasound probe (e.g., an area or solid angle covered by the ultrasound data).

[0022] The ultrasound data may indicate intensities of reflections of the respective ultrasound signals and / or corresponding delays after emission of the respective ultrasound signals. The ultrasound data may indicate respective directions from which the ultrasound sensor received the reflections. The ultrasound data may represent a (two-dimensional (2D) or three-dimensional (3D)) image generated based on the intensities and / or delays and / or the respective directions.

[0023] A description of further details of obtaining ultrasound data is omitted because ultrasound imaging is generally known in the art and one skilled in the art will find suitable techniques for obtaining the ultrasound data.

[0024] Accordingly, when a user places the ultrasound probe on the mother's abdomen, the obtained ultrasound data can represent the fetus in the mother's uterus.

[0025] Creating the data model of the fetus may include generating a three-dimensional (3D) representation of the fetus. For example, the data model may include a 3D spline representation of the fetus. For example, the data model may include a tessellated representation of the fetus and may, for example, specify vertices and / or (triangular or polygonal) facets representing the fetus. For example, the data model may include a 3D point cloud and / or a 3D mesh representing the fetus. The data model is not limited to these types, and one of ordinary skill in the art will find other suitable representations of the fetus in a data model.

[0026] Creating the data model may involve identifying the fetus (or part of the fetus) in the ultrasound data and omitting other structures (e.g., the uterus, bones, or other maternal tissue) represented by the ultrasound data, so that the data model can represent the fetus in isolation from the other structures. Alternatively, the other structures may be included in the data model.

[0027] Generating the fetus visualization may involve inputting the data model into the generative artificial intelligence. The generative artificial intelligence (AI) may include or be based on an artificial neural network, e.g., a deep neural network, a convolutional neural network, a neural radiance field (NeRF), a variational autoencoder (VAE), a generative adversarial network (GAN), or the like. For example, the generative AI may use or be similarly configured to use DALL-E, Midjourney, Stable Diffusion, Deep Dream Generator, StyleGAN, BigGAN, or the like.

[0028] The generative AI may be configured (e.g., trained) to generate a detailed image of the fetus based on an input that includes the data model and / or the ultrasound data. For example, the generative AI may remove noise or artifacts from the ultrasound data that may still be present in the data model. The generative AI may generate the visualization of the fetus such that a pose of the fetus is visualized according to the ultrasound data. The visualization of the fetus may include a 2D or 3D image of the fetus. The generative AI may include a 3D visualization of the fetus, where the fetus is shown in a projection as viewed from a predefined perspective and / or where an associated shadow of the fetus is shown.The visualization of the fetus can be generated as a grayscale image and / or as a color image, where colors of the color image can correspond to true / realistic colors or pseudo-colors that can indicate respective predefined anatomical features (e.g., types of tissue, body parts of the fetus, deviation of an anatomical feature from a typical value, differentiation between the fetus and the mother, or the like).

[0029] The circuitry can generate the visualization of the fetus in real time. For example, the ultrasound probe can generate ultrasound data based on respective ultrasound signals at different times (e.g., at a predefined rate; e.g., every second, every 100 ms, every 10 ms, every millisecond, every 0.2 milliseconds, or the like, without limiting the disclosure to these values). The circuitry can obtain the respective most recent ultrasound data, create a data model of the fetus based on the respective most recent ultrasound data (or based on ultrasound data that is newer than the last update of the data model) (e.g., update a data model created based on previous ultrasound data), and generate a visualization of the fetus based on the respective most recent (e.g., updated) data model (e.g.,update a visualization generated based on a previous data model). For example, the circuitry may generate an updated visualization of the fetus at a predefined frame rate (e.g., 10 frames per second (fps), 30 fps, 50 fps, 100 fps, or the like, without limiting the disclosure to these values).

[0030] The circuitry can write the generated visualization of the fetus to storage (e.g., a hard disk drive (HDD), a solid-state drive (SSD), a USB flash drive, a Secure Digital (SD) card, or the like), upload the generated visualization to a server (e.g., via Ethernet or Wi-Fi), and / or send the generated visualization to a display device for display (e.g., via Video Graphics Array (VGA), High-Definition Multimedia Interface (HDMI), DisplayPort, USB, Wi-Fi, Bluetooth, or the like). For example, the circuitry can send the visualization of the fetus to a screen in the office of a physician (e.g., a gynecologist) performing an ultrasound examination. For example, the circuitry can send the visualization of the fetus to a portable or wearable device (e.g.,A user (e.g., the physician performing the ultrasound examination, the mother or father of the fetus) can receive a signal from a device (e.g., a smartphone, a head-mounted display, data glasses, a smartwatch, or the like). The display device can display the visualization of the fetus.

[0031] Accordingly, some embodiments provide for the use of ultrasound scan information (e.g., ultrasound data) combined with generative AI to provide a precise understanding of a condition of a baby (e.g., the fetus) within the mother's womb (e.g., the uterus). If a well-defined environment and structure of the fetus / baby within the mother's womb are present, allowing the capture of certain details, such as body parts and developing organs such as the brain, heart, spine, etc., a model (e.g., the data model) may be constructed based on an initial scan using the ultrasound probe. A thorough ultrasound scan from multiple directions may be required to capture sufficient detail and create the data model of the fetus / baby.Once enough details have been observed, the visualization of the fetus / baby can be created by adding skeletal structure, current posture, size, and position in the womb.

[0032] Using the created data model, the fetus can be visualized in real time, which may include adjusting and animating the visualization of the fetus using real-time information from position changes, a detected heartbeat, and other aspects and vital signs of the fetus that can be determined based on the ultrasound data. The visualization can be displayed on a 2D screen, a mobile device, or through an augmented reality (AR) / virtual reality (VR) device.

[0033] Accordingly, the circuitry may be configured to perform AR visualization of the fetus / baby in the uterus / womb.

[0034] In some embodiments, the circuitry is further configured to: obtain image data of a portion of an abdomen of the mother; and determine a position of the fetus relative to the mother; and generating the visualization of the fetus includes generating the visualization according to the determined position.

[0035] The circuitry can determine the position of the fetus relative to the mother based on the ultrasound data and the image data. The circuitry can generate the visualization such that the visualization displays the fetus in a posture and / or orientation that corresponds to a posture and / or orientation of the fetus in the uterus.

[0036] For example, generating the visualization of the fetus may involve inserting the visualization of the fetus into an image represented by the image data according to the determined position of the fetus. The visualization may show the fetus within the mother's abdomen, e.g., the fetus framed by the mother's abdomen.

[0037] For example, the fetus can be "anchored" to a physical position of the mother. The fetus's position can be observed in the ultrasound scan and anchored relative to external features of the abdomen specified in the image data. In this way, the generated data model of the fetus and the precise position of the fetus within the womb / uterus can be visualized in real time, thus providing the user with a precise understanding of the fetus's position, posture, movements, etc.

[0038] For example, the circuitry can perform accurate 3D reconstruction and high-quality generation of the data model of the fetus / baby including segmented body parts, which can be animated.

[0039] In some embodiments, determining the position of the fetus includes: identifying, in the ultrasound data, a structure of a body of the mother; and determining, in an image represented by the image data, a position corresponding to the structure.

[0040] For example, the circuitry may identify a bone (e.g., hip, vertebra, rib, etc.), a tissue, a muscle, an organ, a blood vessel, the uterus, or the like as a structure of the mother's body in the ultrasound data. The circuitry may determine an orientation of the mother in the image data, e.g., based on a well-known position of a structure of an outer region of the mother's abdomen (such as the navel, hips, rib cage, etc.). Based on the determined orientation of the mother in the image data, the circuitry may match a position of the structure identified in the ultrasound data to a position in the image data, e.g., according to a typical anatomical condition known in the art.

[0041] The circuitry can generate the visualization such that an orientation of the fetus in the visualization relative to the mother's abdomen corresponds to an actual orientation of the fetus in the uterus relative to the mother's abdomen. For example, the circuitry can orient the data model of the fetus accordingly and generate the visualization based on the oriented data model.

[0042] In some embodiments, the circuitry is further configured to determine a position of a display device relative to the mother; and generating the visualization of the fetus includes generating the visualization of the fetus in a perspective as viewed from the position of the display device.

[0043] The image data of the mother's abdomen can be acquired using a camera of a mobile device (e.g., smartphone, tablet, notebook, head-mounted display, smart glasses, smartwatch, AR / VR device, etc.). A user can point the mobile device's camera at the mother's abdomen. The mobile device can further include a display device (e.g., liquid crystal display (LCD), organic light-emitting diode (OLED), quantum dot display (QLED), plasma display panel (PDP), cathode ray tube (CRT), or the like). The circuitry can send the generated visualization of the fetus to the mobile device, and the mobile device can cause its display device to display the visualization of the fetus.

[0044] A positional relationship between the camera of the mobile device and the display device of the mobile device may be predefined (e.g., the camera may be oriented in an opposite direction to the display device, without limiting the disclosure to such a configuration). The circuitry may then determine that the perspective as viewed from the position of the display device corresponds to a perspective as viewed through the camera. The circuitry may determine the position of the display device relative to the mother (e.g., relative to the mother's abdomen or a portion of the abdomen represented in the image data) based on the image data (e.g., based on a perspective from which the abdomen is represented in the image data).

[0045] When a user moves the mobile device with the camera relative to the mother's abdomen, the circuitry may update the visualization of the fetus (e.g., a perspective from which the fetus is visualized) according to the changed position of the camera.

[0046] For example, the fetus can be anchored relative to the mother's abdomen using a mobile device (e.g., AR device, smartphone, smart glasses, etc.). For example, a user can point the mobile (portable / wearable) device directly at the mother's abdomen for visualization, providing a real-time view "inside" the womb. This allows for precise visualization of the exact scale and posture, thus providing a true understanding and sense of the fetus's / baby's condition. Similarly, visualization of the fetus using an AR device may be possible to "look inside" the mother's abdomen through a physical barrier and better capture the real-time position and movements of the fetus / baby.

[0047] The image data may represent a grayscale image, a color (e.g., red-green-blue (RGB)) image, a depth image (e.g., obtained based on time-of-flight (ToF) imaging), an infrared image, or the like.

[0048] However, the disclosure is not limited to obtaining the image data with a mobile device that displays the visualization of the fetus. For example, the camera may be provided separately from the mobile device and / or display device, and a positional relationship between the display device and the camera may be determined based on, for example, acoustic ranging, radio ranging, and / or identifying the display device in the image data.

[0049] In some embodiments, generating the visualization of the fetus includes reconstructing a missing portion of the fetus, wherein the missing portion is a portion not indicated by the ultrasound data.

[0050] The missing part of the fetus may include a part of the fetus that could not be detected in the ultrasound data due to an orientation and / or posture of the fetus (e.g., the fetus may "hide" a body part from ultrasound acquisition), a position of the ultrasound probe (e.g., a body part of the fetus may be outside a field of view of the ultrasound probe), and / or a penetration depth of the ultrasound signal (e.g., the ultrasound signal may not reach a body part (e.g., an organ or bone) of the fetus with sufficient signal strength).

[0051] Generative AI can be trained based on a typical fetal anatomy to reconstruct the missing part of the fetus, allowing visualization of all body parts of the fetus and / or from any perspective.

[0052] Accordingly, if some parts of the fetus could not be captured due to the position of the fetus within the womb and / or ultrasound probe positioning, such parts can be automatically generated by generative AI using generative AI methods such as NeRF or similar.

[0053] The circuitry can indicate reconstructed parts of the fetus as generated rather than directly captured to ensure visualization quality, so that, for example, a doctor can recognize from the visualization that a body part of the fetus was reconstructed by generative AI.

[0054] The circuitry may indicate the reconstructed parts as generated based on a color (e.g., the reconstructed parts may be visualized in a different color than observed parts of the fetus) and / or based on a frame, label, hatching, or the like added to the visualization. A user may switch between a raw mode and an advanced mode. The raw mode may be a mode in which the reconstructed parts are indicated as generated or in which reconstructing parts of the fetus is disabled (e.g., for a physician examining the fetus), and the advanced mode may be a mode in which the missing parts are reconstructed without an indication of which parts are reconstructed (e.g., for parents of the fetus who want to see an image of their baby).

[0055] In some embodiments, the missing part includes an organ of the fetus. Generally, in some embodiments, the missing part includes external and / or internal features of the fetus, such as skin, hair, eyes, bones, muscles, organs, etc.

[0056] Generative AI can be used to further expand a view to reconstruct a realistic view of the fetus, including realistic renderings of the fetus's external and internal features, such as skin, hair, eyes, bones, muscles, and organs. Such information may not only be useful for expectant parents to "meet" their baby, but can also provide more accurate information for medical professionals to examine the health and condition of the fetus / baby and take more precise measurements.

[0057] In some embodiments, the ultrasound data represents the fetus at two different points in time; and the circuitry is configured to interpolate the fetus for a point in time between the two different points in time.

[0058] Interpolating the fetus may involve interpolating a state of the fetus, e.g., an appearance, size, posture, orientation, developmental stage, etc. of the fetus. The interpolation may be performed using generative AI. The circuitry may generate a visualization of the interpolated (state of the) fetus as a video, as a sequence of images, or the like. For example, the video or sequence of images may visualize a change in the state of the fetus as a smooth transition and / or as a sequence of developmental stages. The two different time points may correspond to times at which an ultrasound examination of the fetus is performed.

[0059] For example, during subsequent visits to a doctor, a growth “timeline” of the fetus / baby can be created and fetal development can be accurately tracked over time.

[0060] Accordingly, based on multiple ultrasound scans (and resulting ultrasound data; e.g., ultrasound data from different time points) and the data model of the fetus / baby in the womb, the baby's growth can be interpolated for unaccounted days between the ultrasound scans. By incorporating data from all ultrasound scans, a complete developmental cycle can be interpolated (and potentially extrapolated as needed) to show the growth timeline and capture characteristics of the specific fetus / baby. Any number of ultrasound scans can be used to generate the visualization of the fetus according to the growth timeline.

[0061] The result of generating the growth timeline can be specific to the observed fetus / baby and can present a precise capture of the fetal anatomy during ultrasound scans and an interpolation of intermediate steps using generative AI. Visualization of the growth timeline can be possible from any angle. Both doctors and parents can benefit from observing the fetus / baby's development in detail.

[0062] In some embodiments, the data model includes a three-dimensional (3D) representation of the fetus. The 3D representation of the fetus in the data model may enable the creation of a visualization of the fetus from any direction.

[0063] In some embodiments, the ultrasound data represents the fetus in respective perspectives, such as when viewed from multiple directions.

[0064] For example, the visualization of the fetus can be based on ultrasound data from different ultrasound scans, e.g., ultrasound data obtained from different positions in the mother's abdomen. Accordingly, the data model (and visualization) of the fetus can indicate fetal features that cannot be observed simultaneously within a single ultrasound scan.

[0065] In some embodiments, the circuitry is further configured to generate an instruction to position an ultrasonic sensor on the mother's abdomen.

[0066] The circuitry can cause the instruction to be issued to a user (e.g., a physician) performing an ultrasound examination of the fetus. The instruction can specify a direction in which the ultrasound probe should be moved or rotated so that a feature of the fetus can be observed that is not represented in previously obtained ultrasound data.

[0067] The instruction may include an audible message (e.g., a voice output and / or a beep) and may be output, for example, through a speaker provided with and / or associated with the ultrasound probe. The instruction may include a visual message (e.g., a text message, an arrow, a light-emitting diode (LED), or the like) and may be output, for example, through the ultrasound probe and / or through the display device displaying the visualization.

[0068] Furthermore, while acquiring ultrasound data, the circuitry can guide the positioning of the ultrasound probe to estimate how it should be held to capture "unseen" parts of the current ultrasound scan. This can enable not only medical professionals but also casual users, such as expectant parents or their relatives, to perform an ultrasound scan at home.

[0069] The circuitry may determine a current position of the ultrasound probe based on the ultrasound data, for example, based on a perspective from which the fetus is represented in the ultrasound data, based on structures of the mother's body represented in the ultrasound data (e.g., a bone (e.g., hip, vertebra, rib, etc.), a tissue, a muscle, an organ, a blood vessel, the uterus, or the like), and / or based on a representation of the ultrasound probe in image data of the mother's abdomen. The circuitry may use the determined position of the ultrasound probe for guidance to arrive at a precise relative position between the ultrasound probe (and / or a mobile device, as previously described) and the baby / fetus in the womb.

[0070] Some embodiments relate to a method for generating a visualization of a fetus, the method comprising: obtaining ultrasound data representing a fetus in a uterus of its mother; creating, based on the ultrasound data, a data model of the fetus; and generating, based on the data model, a visualization of the fetus using generative artificial intelligence.

[0071] The method may be configured similarly to the previously described circuit arrangement, and any feature described with reference to the circuit arrangement may correspond to a respective feature of the method. The features previously described with reference to the circuit arrangement and / or the method may be combined in any suitable manner.

[0072] The methods described herein are also implemented, in some embodiments, as a computer program that causes a computer and / or processor to perform the method when executed on the computer and / or processor. In some embodiments, a non-transitory computer-readable recording medium is also provided having stored therein a computer program product that, when executed by a processor, such as the processor described above, causes the methods described herein to be performed.

[0073] Returning to Fig. 1 illustrates Fig. 1 shows an embodiment of a circuit arrangement 1. The circuit arrangement 1 includes a processing unit 2, an artificial intelligence (AI) processing unit 3, a memory unit 4, and a communication unit 5. The processing unit 2, the AI ​​processing unit 3, the memory unit 4, and the communication unit 5 can communicate with each other via a bus.

[0074] The processing unit 2 includes a central processing unit (CPU) and is configured to perform general information processing and control the AI ​​processing unit 3, the storage unit 4 and the communication unit 5.

[0075] The AI ​​processing unit 3 includes a graphics processing unit (GPU) and is configured to execute a generative AI 3a. The generative AI 3a is based on a neural radiation field (NeRF) and is configured to generate a 3D visualization of a fetus 12 based on a 3D data model of the fetus 12 and ultrasound data of the fetus.

[0076] The storage unit 4 includes a dynamic random access memory (DRAM) and a solid-state drive (SSD) and is configured to store instructions for the processing unit 2, parameters of the generative AI 3a, ultrasound data obtained by the circuitry 1, a data model of the fetus 12 created by the circuitry 1, a visualization of the fetus 12 generated by the generative AI 3a, and other data required for a function of the circuitry 1.

[0077] A mother's abdomen 10 contains a uterus 11, and the uterus 11 contains the fetus 12. An ultrasound probe 13 acquires ultrasound data of the fetus 12 in the uterus 11 and transmits the ultrasound data to the communication unit 5 of the circuit arrangement 1 via a USB connection 13a.

[0078] The processing unit 2 receives the ultrasound data representing the fetus 12 in the uterus 11 from the ultrasound probe 13 and creates a data model of the fetus 12 based on the ultrasound data. The AI ​​processing unit 3 then generates a visualization of the fetus 12 based on the data model using the generative AI 3a.

[0079] The communication unit 5 sends the generated visualization of the fetus 12 to a screen 14 (e.g., a display unit) via a high-definition multimedia interface (HDMI) connection 14a. The screen 14 is mounted in the office of a physician performing an ultrasound examination of the fetus 12. The screen 14 displays the visualization of the fetus 12 so that the physician can examine the fetus 12.

[0080] A smartphone 15 (an example of a mobile device) acquires image data representing the mother's abdomen 10 using a camera provided in the smartphone 15 and transmits the image data to the communication unit 5 of the circuit arrangement 1 via a Wi-Fi connection 15a.

[0081] The AI ​​processing unit 3 generates a visualization of the fetus 12 in which the fetus 12 is shown from a perspective of the smartphone 15 (and its camera) in an orientation corresponding to an orientation of the abdomen 10 in the image data, and wherein an image of the fetus 12 is inserted into an image of the abdomen 10 represented by the image data.

[0082] The communication unit 5 sends the visualization to the smartphone 15 via the Wi-Fi connection 15a, and the smartphone 15 causes a display (an example of a display unit) of the smartphone 15 to display the visualization so that parents (i.e., the mother and a father) of the fetus 12 can view the fetus 12 on the smartphone 15 as if they were “looking through” the mother’s abdomen 10.

[0083] The circuit arrangement 1 and its units 2 to 5 are configured to carry out the method described with reference to Fig. 2 is described.

[0084] Fig. 2 illustrates an embodiment of a method 20. The method 20 of Fig. 2 is an example of a method which is implemented by the circuit arrangement 1 of Fig. 1 and its units 2 to 5.

[0085] At 21 of the method 20, the processing unit 2 receives ultrasound data representing the fetus 12 in the uterus 11 of its mother via the communication unit 5 from the ultrasound probe 13. The ultrasound data represent the fetus 12 in respective perspectives when viewed from multiple directions.

[0086] At step 22 of the method 20, the processing unit 2 creates a data model of the fetus 12 based on the ultrasound data obtained at step 21. The data model includes a 3D representation of the fetus 12.

[0087] At 23 of the method 20, the processing unit 2 receives image data of a part of the mother's abdomen 10 from the smartphone 15 via the communication unit 5.

[0088] At 24 of the method 20, the processing unit 2 determines a position of the fetus 12 relative to the mother (i.e., relative to the mother's abdomen 10). Determining the position of the fetus 12 at 24 includes identifying, at 24a, a structure of a body of the mother in the ultrasound data and determining, at 24b, a position corresponding to the structure in an image represented by the image data.

[0089] At 25 of the method 20, the processing unit 2 determines a position of the smartphone 15 (and accordingly the display of the smartphone 15, which is an example of a display device) relative to the mother (ie, relative to the abdomen 10) based on the image data (ie, based on a perspective from which the abdomen 10 is represented in the image data).

[0090] At 26 of the method 20, the AI ​​processing unit 3 generates a visualization of the fetus 12 using the generative AI 3a based on the data model created at 22, based on the ultrasound data obtained at 21, and based on the image data obtained at 23, wherein generating the visualization includes executing the generative AI 3a. The AI ​​processing unit 3 generates the visualization according to the determined position of the fetus 12 determined at 24 and according to the predetermined position of the display device 15 determined at 25, in a perspective as viewed from the position of the display device 15.

[0091] Creating the visualization of the fetus 12 at 26 includes reconstructing, at 26a, a missing portion of the fetus 12, wherein the missing portion is a portion of the fetus 12 not indicated by the ultrasound data. The missing portion includes an organ of the fetus 12 (without limiting the disclosure to reconstructing an organ; the missing portion, in some embodiments, may include any external and / or internal feature of the fetus 12, such as skin, hair, eyes, bones, muscles, organs, etc.).

[0092] Generating the visualization of fetus 12 at 26 includes interpolating, at 26b, (a state of) fetus 12 for a time between two different time points using generative AI 3a, where the two different time points correspond to times at which an ultrasound examination of fetus 12 is performed, and the ultrasound data represents the (state of) fetus 12 at two different time points. Interpolating at 26b includes generating a video showing a smooth transition between the state of fetus 12 at the two different time points, such that a growth timeline of fetus 12 is visualized.

[0093] At step 27 of the method 20, the processing unit 2 generates an instruction for positioning an ultrasound sensor of the ultrasound probe 13 on the mother's abdomen 10. The instruction specifies a position to which the ultrasound probe 13 (with the ultrasound sensor) should be moved in order to obtain ultrasound data of a part of the fetus 12 that is not yet represented in sufficient detail in the ultrasound data. The processing unit 2 causes both the screen 14 and a light-emitting diode (LED)-based display of the ultrasound probe 13 to display an arrow corresponding to a direction in which the ultrasound probe 13 should be moved.

[0094] At 28, the communication unit 5 sends the visualization of the fetus 12 generated at 26 to a display device. Sending at 28 includes sending the visualization to the screen 14 so that a doctor performing an ultrasound examination of the fetus 12 can examine the fetus 12. Sending at 28 also includes sending the visualization to the smartphone 15 so that parents of the fetus 12 can view the fetus 12. Sending at 28 also includes sending the video generated at 26b, which visualizes the growth timeline of the fetus 12, to the smartphone 15. The smartphone 15 saves the video so that the parents of the fetus 12 can view the video again at a later time.

[0095] As in Fig. 2, indicated by the arrow from 28 to 21, the circuit arrangement 1 executes a next pass of the method 20 so that the fetus 12 can be visualized in real time (e.g., at a frame rate of 30 fps, without limiting the disclosure to this value). In the next pass of the method 20, the processing unit 2 receives more recent ultrasound data at 21, updates the data model at 22, receives more recent image data at 23, determines an updated position of the fetus 12 at 24, determines an updated position of the display device 15 at 25, generates an updated visualization of the fetus 12 at 26, generates an updated instruction for positioning the ultrasound probe 13 at 27, and sends the updated visualization of the fetus 12 to the display devices 14 and 15 at 28.The circuit arrangement 1 executes a next pass of the method 20 during an ongoing ultrasound examination and stops executing a next pass of the method 20 when a user stops the ultrasound examination.

[0096] It is noted that in some embodiments, one or more parts of the method 20 may be Fig. 2 can be omitted. In some embodiments, determining the position of the display device 15 at 25 is omitted, and the visualization at 26 is generated regardless of a position of the display device 15 relative to the mother's abdomen 10. Furthermore, in some embodiments, obtaining the image data at 23 and determining the position of the fetus at 24 are also omitted, and the visualization at 26 is generated regardless of a position of the fetus 12 relative to (the abdomen 10) of the mother. In some embodiments, reconstructing a missing part of the fetus 12 at 26a is omitted. In some embodiments, interpolating the (state of the) fetus 12 at 26b is omitted. In some embodiments, generating instruction 27 is omitted.One skilled in the art will find further suitable variations of the method 20, wherein one or more parts of the method 20 may be omitted and / or wherein an order of parts of the method 20 may be changed.

[0097] Fig. 3 illustrates an embodiment of a general-purpose computer 150. The general-purpose computer 150 is an example of an information processing device that includes circuitry configured to perform the method according to the present technology (e.g., the method 20 of Fig. 2). The computer 150 has components 151 to 161 that can form a circuit arrangement, such as the circuit arrangement 1 of Fig. 1, as described here.

[0098] Embodiments that utilize software, firmware, programs, or the like to perform the methods as described herein may be installed on the computer 150, which is then configured to suit the particular embodiment.

[0099] The computer 150 has a CPU 151 (Central Processing Unit) that can execute various types of procedures and methods as described herein, for example, according to programs stored in a read-only memory (ROM) 152, stored in a memory 157 and loaded into a random access memory (RAM) 153, stored on a medium 160 that can be inserted into a respective drive 159, etc.

[0100] Furthermore, the computer 150 includes an artificial intelligence (AI) processor 151a. The AI ​​processor 151a may include a graphics processing unit (GPU) and / or a tensor processing unit (TPU). The AI ​​processor 151a may be configured to execute an AI model (e.g., an artificial neural network), for example, the generative AI 3a of Fig. 1, be configured.

[0101] The CPU 151, the ROM 152, and the RAM 153 are connected to a bus 161, which in turn is connected to an input / output interface 154. The number of CPUs, memories, and storage is only exemplary, and one skilled in the art will understand that the computer 150 can be configured and adapted to meet specific requirements that arise when functioning as a base station or as a user device (terminal).

[0102] Several components are connected to the input / output interface 154: an input 155, an output 156, the storage 157, a communication interface 158 and the drive 159 into which a medium 160 (compact disc (CD), digital video disc (DVD), universal serial bus (USB) flash drive, secure digital (SD) card, CompactFlash (CF) memory or the like) can be inserted.

[0103] The input 155 may be a pointing device (mouse, graphics tablet, or the like), a keyboard, a microphone, a camera, a touch screen, an eye tracking unit, etc.

[0104] The output 156 may include a display (liquid crystal display (LCD), cathode ray tube (CRT) display, light-emitting diode (LED) display, electronic paper, etc.; e.g., included in a touch screen), speakers, etc.

[0105] The storage 157 may include a hard disk drive (HDD), a solid state drive (SSD), a flash drive, and the like.

[0106] The communication interface 158 may be configured to communicate via, for example, Universal Serial Bus (USB), a serial port (RS-232), a parallel port (IEEE 1284), a local area network (LAN; e.g., Ethernet), a wireless local area network (WLAN; e.g., Wi-Fi, IEEE 802.11), a mobile telecommunications system (GSM, UMTS, LTE, NR, etc.), Bluetooth, near field communication (NFC), ZigBee, infrared, etc.

[0107] It should be noted that the foregoing description relates only to an exemplary configuration of computer 150. Alternative configurations may be implemented with additional or different sensors, storage devices, interfaces, or the like. For example, communications interface 158 may support radio access technologies other than those mentioned, such as UMTS, LTE, and NR.

[0108] It is understood that the embodiments describe methods with an exemplary order of method steps. However, the specific order of the method steps is provided for illustrative purposes only and should not be construed as binding. For example, the order of Figures 21 and 22 in the embodiment of Figure 1 may Fig. 2. In addition, the order of 24 and 25 in the embodiment of Fig. 2. Furthermore, 23, 24 and / or 25 can be placed in front of 22 or 21 in the embodiment of Fig. 2. Furthermore, the sequence of 26 and 27 in the embodiment of Fig. 2 may be reversed. Other changes in the sequence of process steps may be apparent to a person skilled in the art.

[0109] It should be noted that the division of circuitry 1 into units 2 to 5 is for illustrative purposes only, and that the present disclosure is not limited to any specific division of functions into specific units. For example, circuitry 1 could be implemented by a respective programmed processor, a field-programmable gate array (FPGA), and the like.

[0110] The procedure from Fig.2 can also be implemented as a computer program that causes a computer and / or a processor, such as the previously discussed circuit arrangement 1, to perform the method when executed on the computer and / or the processor. In some embodiments, a non-transitory computer-readable recording medium is also provided in which a computer program product is stored that, when executed by a processor, such as the previously described processor, causes the described method to be performed.

[0111] All units and entities described in this specification and claimed in the appended claims may, unless otherwise stated, be implemented as integrated circuit logic, for example on a chip, and functionality provided by such units and entities may, unless otherwise stated, be implemented by software.

[0112] To the extent that the above-described embodiments of the disclosure are implemented at least in part using a software-controlled data processing device, it is to be understood that a computer program providing such software control and a transmission, storage, or other medium through which such a computer program is provided are contemplated as aspects of the present disclosure.

[0113] It is noted that the present technology may also be configured as described below. (1) A circuit arrangement for generating a visualization of a fetus, the circuit arrangement being configured to: Obtaining ultrasound data representing a fetus in its mother's uterus; Creating a data model of the fetus based on the ultrasound data; and Generate, based on the data model, a visualization of the fetus using generative artificial intelligence. (2) The circuit arrangement according to (1), wherein the circuit arrangement is further configured to: Obtaining image data of a portion of a mother’s abdomen; and Determining a position of the fetus relative to the mother; and wherein generating the visualization of the fetus includes generating the visualization according to the determined position. (3) The circuit arrangement according to (2), wherein determining the position of the fetus includes: Identifying, in the ultrasound data, a structure of a mother’s body; and Determine, in an image represented by the image data, a position corresponding to the structure. (4) The circuit arrangement according to (2) or (3), wherein the circuit arrangement is further configured to determine a position of an indicator device relative to the nut; and wherein generating the visualization of the fetus includes generating the visualization of the fetus in a perspective as viewed from the position of the display device. (5) The circuit arrangement according to any one of (1) to (4), wherein generating the visualization of the fetus includes reconstructing a missing part of the fetus, the missing part being a part not indicated by the ultrasound data. (6) The circuit arrangement according to (5), the missing part contains an organ of the fetus. (7) The circuit arrangement according to any one of (1) to (6), where the ultrasound data represent the fetus at two different time points; and wherein the circuitry is configured to interpolate the fetus for a time between the two different time points. (8) The circuit arrangement according to any one of (1) to (7), where the data model includes a three-dimensional representation of the fetus. (9) The circuit arrangement according to any one of (1) to (8), where the ultrasound data represent the fetus in respective perspectives when viewed from multiple directions. (10) The circuit arrangement according to any one of (1) to (9), wherein the circuitry is further configured to generate an instruction for positioning an ultrasonic sensor on the mother's abdomen. (11) A method for generating a visualization of a fetus, the method comprising: Obtaining ultrasound data representing a fetus in its mother's uterus; Creating a data model of the fetus based on the ultrasound data; and Generate, based on the data model, a visualization of the fetus using generative artificial intelligence. (12) The procedure under (11), the method further comprising: Obtaining image data of a portion of a mother’s abdomen; and Determining a position of the fetus relative to the mother; and wherein generating the visualization of the fetus includes generating the visualization according to the determined position. (13) The method of (12), wherein determining the position of the fetus comprises: Identifying, in the ultrasound data, a structure of a mother’s body; and Determine, in an image represented by the image data, a position corresponding to the structure. (14) The procedure under (12) or (13), wherein the method further comprises determining a position of an indicator device relative to the nut; and wherein generating the visualization of the fetus includes generating the visualization of the fetus in a perspective as viewed from the position of the display device. (15) The procedure under any of (11) to (14), wherein generating the visualization of the fetus includes reconstructing a missing part of the fetus, the missing part being a part not indicated by the ultrasound data. (16) The procedure under (15), the missing part contains an organ of the fetus. (17) The procedure under any of (11) to (16), where the ultrasound data represent the fetus at two different time points; and wherein the method comprises interpolating the fetus for a time point between the two different time points. (18) The procedure under any of (11) to (17), where the data model includes a three-dimensional representation of the fetus. (19) The procedure under any of (11) to (18), where the ultrasound data represent the fetus in respective perspectives when viewed from multiple directions. (20) The procedure under any of (11) to (19), wherein the method further comprises generating an instruction for positioning an ultrasound sensor on the abdomen of the mother. (21) A computer program comprising program code which, when executed on a computer, causes a computer to perform the method of any one of (11) to (20). (22) A non-transitory computer-readable recording medium storing therein a computer program product which, when executed by a processor, causes the method of any one of (11) to (20) to be performed. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited non-patent literature

[0000] Peripheral Component Interconnect (PCI), serial port (RS-232), parallel port (IEEE 1284), Ethernet, wireless local area network (WLAN; e.g. Wi-Fi, IEEE 802.11

[0016]

Claims

[1] Circuitry for generating a visualization of a fetus, the circuitry being configured to: Obtaining ultrasound data representing a fetus in its mother's uterus; Creating a data model of the fetus based on the ultrasound data; and Generate, based on the data model, a visualization of the fetus using generative artificial intelligence. [2] The circuit arrangement of claim 1, wherein the circuit arrangement is further configured to: Obtaining image data of a portion of a mother’s abdomen; and Determining a position of the fetus relative to the mother; and wherein generating the visualization of the fetus includes generating the visualization according to the determined position. [3] The circuit arrangement of claim 2, wherein determining the position of the fetus includes: Identifying, in the ultrasound data, a structure of a mother’s body; and Determine, in an image represented by the image data, a position corresponding to the structure. [4] Circuit arrangement according to claim 2, wherein the circuit arrangement is further configured to determine a position of an indicator device relative to the nut; and wherein generating the visualization of the fetus includes generating the visualization of the fetus in a perspective as viewed from the position of the display device. [5] The circuit arrangement of claim 1, wherein generating the visualization of the fetus includes reconstructing a missing part of the fetus, the missing part being a part not indicated by the ultrasound data. [6] A circuit arrangement according to claim 5, wherein the missing part includes an organ of the fetus. [7] Circuit arrangement according to claim 1, where the ultrasound data represent the fetus at two different time points; and wherein the circuitry is configured to interpolate the fetus for a time between the two different time points. [8] The circuit arrangement of claim 1, wherein the data model includes a three-dimensional representation of the fetus. [9] The circuit arrangement of claim 1, wherein the ultrasound data represents the fetus in respective perspectives when viewed from multiple directions. [10] The circuit arrangement of claim 1, wherein the circuit arrangement is further configured to generate an instruction to position an ultrasonic sensor on the abdomen of the mother. [11] A method for generating a visualization of a fetus, the method comprising: Obtaining ultrasound data representing a fetus in its mother's uterus; Creating a data model of the fetus based on the ultrasound data; and Generate, based on the data model, a visualization of the fetus using generative artificial intelligence. [12] The method of claim 11, further comprising: Obtaining image data of a portion of a mother’s abdomen; and Determining a position of the fetus relative to the mother; and wherein generating the visualization of the fetus includes generating the visualization according to the determined position. [13] The method of claim 12, wherein determining the position of the fetus includes: Identifying, in the ultrasound data, a structure of a mother’s body; and Determine, in an image represented by the image data, a position corresponding to the structure. [14] Method according to claim 12, wherein the method further comprises determining a position of an indicator device relative to the nut; and wherein generating the visualization of the fetus includes generating the visualization of the fetus in a perspective as viewed from the position of the display device. [15] The method of claim 11, wherein generating the visualization of the fetus includes reconstructing a missing part of the fetus, wherein the missing part is a part not indicated by the ultrasound data. [16] The method of claim 15, wherein the missing part includes an organ of the fetus. [17] Method according to claim 11, where the ultrasound data represent the fetus at two different time points; and wherein the method comprises interpolating the fetus for a time point between the two different time points. [18] The method of claim 11, wherein the data model includes a three-dimensional representation of the fetus. [19] The method of claim 11, wherein the ultrasound data represents the fetus in respective perspectives when viewed from multiple directions. [20] The method of claim 11, wherein the method further comprises generating an instruction for positioning an ultrasonic sensor on the abdomen of the mother.

Images