Multiple gestation detection using ultrasound imaging with a blind sweep protocol

Abstract

A system includes a processor configured for communication with an ultrasound probe, wherein the processor is configured to: control the ultrasound probe to obtain a plurality of ultrasound image frames during a blind sweep protocol on a pregnant patient; provide the plurality of ultrasound image frames as input to a deep learning network trained to detect at least one of maternal anatomy or fetal anatomy; and generate a detection result as output of the deep learning network. The detection result includes at least one of: a twin inter-twin membrane in the plurality of ultrasound image frames, or a plurality of fetal anatomical sites in the plurality of ultrasound image frames. The processor is further configured to: utilize the detection result to determine whether the pregnancy includes a multiple pregnancy; and provide output representative of the determination to a display in communication with the processor.

Description

Technical Field

[0001] The objects described herein relate to devices, systems, and methods for detecting multiple pregnancies / gestations using ultrasound data from blind abdominal imaging scans. Background Technology

[0002] Ultrasound imaging is typically used for diagnostic purposes in office or hospital settings, but it may also be used in resource-constrained care settings (e.g., homes, accident scenes, ambulances, mobile medical facilities, etc.) by emergency responders, home healthcare nurses, midwives, etc., who may lack ultrasound expertise. To facilitate the acquisition of ultrasound images by untrained or minimally trained users, a "blind scan" protocol is often employed, in which the user scans along a pre-defined probe path (e.g., sweeping a pattern across the patient's abdomen) during imaging.

[0003] Ultrasound imaging is an essential component of high-quality obstetric care. For example, detecting multiple pregnancies via ultrasound can allow for appropriate referrals to well-resourced medical centers with specialized staff trained to manage any associated risks and complications. However, in rural and resource-poor communities, the scarcity of ultrasound imaging results in a significant gap in maternal health care.

[0004] Multiple pregnancies can pose significant risks to both mother and fetus and are therefore clinically classified as high-risk pregnancies. The diagnosis and management of these pregnancies are challenging. In multiple pregnancies, over 80% of women experience prenatal complications, including preterm birth, premature rupture of membranes, intrauterine growth restriction (IUGR), intrauterine fetal death, gestational diabetes, and preeclampsia, compared to approximately 25% in singleton pregnancies. While ultrasound is extremely useful in identifying multiple pregnancies, access to ultrasound examinations is often limited in resource-scarce environments, and the necessary expertise for identification is also limited. Consequently, the availability and skills required to perform ultrasounds are insufficient in many resource-poor regions of the world.

[0005] The information contained in the background section of this specification, including any references cited herein and their descriptions or discussions, is included for technical reference only and should not be considered as the subject matter defining the scope of this disclosure. Summary of the Invention

[0006] This invention discloses an ultrasound blind-scanning system for detecting multiple pregnancies. Following a blind-scanning protocol (scanning across specific locations on the body without attempting to locate specific anatomical structures), the system detects anatomical features in captured images and uses these features to identify patients who may require referral for evaluation by a human specialist. For example, the system can detect multiple pregnancies (e.g., twins, triplets, etc.), also known as multiple pregnancies. Based on the system's output, pregnant patients can be referred to obstetricians specifically trained to manage multiple pregnancies. Deep learning networks, such as convolutional neural networks, are used to detect anatomical structures (e.g., intertwin septum, fetal head, fetal heart, etc.) of the pregnant patient or one or more fetuses within the patient's uterus. One advantage of the ultrasound blind-scanning anatomical structure detection system is that it uses information acquired during the blind-scanning protocol to identify medical conditions that may require specialist care compared to singleton pregnancies (with only one fetus), thereby improving the quality of care.

[0007] A system of one or more computers can be configured to perform specific operations or actions by installing software, firmware, hardware, or a combination thereof on the system, which, during operation, cause the system to perform the actions. One or more computer programs can be configured to perform specific operations or actions by including instructions that, when executed by a data processing device, cause the device to perform the actions.

[0008] One general aspect includes a system comprising a processor configured to communicate with an ultrasound probe, wherein the processor is configured to: control the ultrasound probe to acquire a plurality of ultrasound image frames during a blind scan protocol on a pregnant patient; provide the plurality of ultrasound image frames as input to a deep learning network trained to detect at least one of maternal or fetal anatomical structures; generate, as output of the deep learning network, a detection of at least one of: a septum between twins in the plurality of ultrasound image frames; or a plurality of fetal anatomical sites in the plurality of ultrasound image frames; determine, using the detection of at least one of the septum between twins or the plurality of fetal anatomical sites, whether the pregnancy is likely to involve a multiple pregnancy; and provide a determined output to a display in communication with the processor, indicating whether the pregnancy is likely to involve a multiple pregnancy. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, all configured to perform operations of the method.

[0009] Implementations may include one or more of the following features. In some aspects, the processor is configured to perform a determination of whether a pregnancy may include a multiple pregnancy based on at least one of the following: detection of an intertwin septum; the simultaneous presence of multiple fetal anatomical sites within a single ultrasound image frame; a geometrical mapping of the multiple fetal anatomical sites; or a geometrical sequence of the multiple fetal anatomical sites. In some aspects, to perform the determination of whether the pregnancy may include a multiple pregnancy, the processor is configured to combine the results of at least two of the following: detection of an intertwin septum; the simultaneous presence of the multiple fetal anatomical sites within the single ultrasound image frame; the geometrical mapping of the multiple fetal anatomical sites; or the geometrical sequence of the multiple fetal anatomical sites. In some aspects, the output of the deep learning network may include multiple detections of the intertwin septum in the plurality of ultrasound image frames, wherein, in order to perform a determination of whether the pregnancy may include a multiple pregnancy based on the detection of the intertwin septum in the plurality of ultrasound image frames, the processor is configured to: determine the number of the plurality of ultrasound image frames having the detection of the intertwin septum; compare the number with a threshold number; and determine that the pregnancy may include a multiple pregnancy when the number exceeds the threshold number. In some aspects, the output provided to the display may include: at least one ultrasound image frame having detected the intertwin septum; and a bounding box identifying the intertwin septum superimposed on the at least one ultrasound image frame. In some aspects, the output of a deep learning network to a single ultrasound image frame may include multiple detections of multiple fetal anatomical sites, wherein, to determine whether a pregnancy may include a multiple pregnancy based on the coexistence of multiple fetal anatomical sites in a single ultrasound image frame, the processor is configured to: take the multiple detections as input to at least one of a statistical model or a rule-based expert system; and generate a determination that the single ultrasound image frame indicates a multiple pregnancy as output to at least one of the statistical model or the rule-based expert system. In some aspects, to determine whether a pregnancy may include a multiple pregnancy based on the coexistence of multiple fetal anatomical sites in a single ultrasound image frame, the processor is configured to: repeat the determination of a multiple pregnancy for the single ultrasound image frame for the multiple ultrasound image frames; determine the number of ultrasound image frames indicating a multiple pregnancy; compare the number to a threshold number; and determine that the pregnancy may include a multiple pregnancy when the number exceeds the threshold number. In some aspects, the output provided to a display may include: a single ultrasound image frame; and multiple bounding boxes superimposed on the single ultrasound image frame identifying the multiple fetal anatomical sites.In some aspects, the output of the deep learning network may include multiple detections of the plurality of fetal anatomical sites in the plurality of ultrasound image frames, wherein, in order to perform a determination of whether the pregnancy may include a multiple pregnancy based on the geometric mapping of the plurality of fetal anatomical sites, the processor is configured to: generate a spatial mapping of the plurality of fetal anatomical sites; and use a rule-based expert system and the spatial mapping to perform a determination that the pregnancy may include a multiple pregnancy. In some aspects, in order to perform the determination of whether the pregnancy may include a multiple pregnancy based on the geometric mapping of the plurality of fetal anatomical sites, the processor is configured to: detect a plurality of regions having the plurality of fetal anatomical sites in the spatial mapping; determine at least one distance between the plurality of regions; provide the plurality of regions and the at least one distance as input to the rule-based expert system; and generate the determination that the pregnancy may include a multiple pregnancy as the output of the rule-based expert system. In some aspects, the output provided to the display may include: the spatial mapping. In some aspects, the output of the deep learning network may include multiple detections of multiple fetal anatomical sites in multiple ultrasound image frames, wherein, in order to perform a determination of whether a pregnancy may include a multiple pregnancy based on the geometric sequence of the multiple fetal anatomical sites, the processor is configured to: generate a text sequence representing anatomical labels of the multiple fetal anatomical sites and the multiple ultrasound image frames; use the text sequence as input to a sequence model; and generate a determination that the pregnancy may include a multiple pregnancy as output of the sequence model. In some aspects, the multiple ultrasound image frames may include a first sweep and a second sweep of a blind scan protocol, wherein the physical location associated with an anatomical label at the end of a first portion of the first sweep in the text sequence is close to the physical location associated with an anatomical label at the beginning of a second portion of the second sweep. In some aspects, the system may include an ultrasound probe. Implementations of the technology may include hardware, methods, or processes, or computer software on a computer-accessible medium.

[0010] One general aspect includes a method comprising: using a processor to control an ultrasound probe communicating with the processor to acquire a plurality of ultrasound image frames during a blind scan protocol on a pregnant patient; using the processor to provide the plurality of ultrasound image frames as input to a deep learning network trained to detect anatomical structures of the patient; using the processor, as output of the deep learning network, generating a detection of at least one of: a septum between twins in the plurality of ultrasound image frames; or a plurality of fetal anatomical sites in the plurality of ultrasound image frames; using the processor, determining whether the pregnancy is likely to include a multiple pregnancy using the detection of at least one of the septum between twins or the plurality of fetal anatomical sites; and using the processor to provide an output indicating whether the pregnancy is likely to include a multiple pregnancy to a display communicating with the processor. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, all configured to perform operations of the method.

[0011] Implementations may include one or more of the following features. In some aspects, determining whether the pregnancy may involve a multiple pregnancy is based on at least one of the following: detection of an intertwin septum; the simultaneous presence of multiple fetal anatomical sites within a single ultrasound image frame; a geometrical mapping of the multiple fetal anatomical sites; or a geometrical sequence of the multiple fetal anatomical sites. In some aspects, the output of the deep learning network may include multiple detections of the intertwin septum in the multiple ultrasound image frames, wherein determining whether the pregnancy may involve a multiple pregnancy based on the detection of the intertwin septum in the multiple ultrasound image frames may include: determining the number of ultrasound image frames with the detection of the intertwin septum; comparing the number to a threshold number; and determining that the pregnancy may involve a multiple pregnancy when the number exceeds the threshold number. In some aspects, the output of a deep learning network to a single ultrasound image frame may include multiple detections of multiple fetal anatomical sites, wherein determining whether a pregnancy is likely to include a multiple pregnancy based on the coexistence of multiple fetal anatomical sites in a single ultrasound image frame may include: providing the multiple detections as input to at least one of a statistical model or a rule-based expert system; and generating a determination that the single ultrasound image frame indicates a multiple pregnancy as the output of at least one of the statistical model or the rule-based expert system. In some aspects, the output of the deep learning network may include multiple detections of multiple fetal anatomical sites in the multiple ultrasound image frames, wherein, in order to determine whether a pregnancy is likely to include a multiple pregnancy based on the geometric mapping of the multiple fetal anatomical sites, the processor is configured to: generate a spatial mapping of the multiple fetal anatomical sites; and use a rule-based expert system and the spatial mapping to perform the determination that a pregnancy may include a multiple pregnancy. In some aspects, the output of a deep learning network may include multiple detections of multiple fetal anatomical sites in multiple ultrasound image frames. To determine whether a pregnancy may include a multiple pregnancy based on the geometric sequence of the multiple fetal anatomical sites, the processor is configured to: generate a text sequence representing the multiple fetal anatomical sites and anatomical labels of the multiple ultrasound image frames; use the text sequence as input to a sequence model; and generate a determination that the pregnancy may include a multiple pregnancy as output to the sequence model. Implementation of the technique may include hardware, methods, or processes, or computer software on a computer-accessible medium.

[0012] This abstract is provided to introduce some concepts in a simplified form, which will be further described in the detailed description section below. This abstract is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. The features, details, uses, and advantages of the ultrasound blind scan multiple pregnancy detection system as defined in the claims are more fully described in the following written description of various aspects of this disclosure, illustrated in the accompanying drawings. Attached Figure Description

[0013] The illustrative aspects of this disclosure will be described with reference to the accompanying drawings, in which...

[0014] Figure 1 This is a schematic diagram of an ultrasound imaging system based on aspects of this disclosure.

[0015] Figure 2 This is a schematic diagram of a processor circuit according to aspects of this disclosure.

[0016] Figure 3A This is a set of schematic cross-sectional views of two fetuses surrounded by an amniotic sac and a chorionic sac within the uterus of a patient, representing different types of twins, based on various aspects of this disclosure.

[0017] Figure 3B It is a set of schematic cross-sectional views of two fetuses located in different positions within a patient's uterus, as observed by an ultrasound imaging plane, based on various aspects of this disclosure.

[0018] Figure 4 This is a schematic diagram of a patient who will undergo a blind ultrasound scan protocol on their abdomen in accordance with various aspects of this disclosure.

[0019] Figure 5 The diagram is a schematic representation of an example ultrasound blind scan multiple pregnancy detection system, presented in the form of a mixed block diagram / flowchart, based on various aspects of this disclosure.

[0020] Figure 6 It is a schematic diagram of at least a portion of an example ultrasound blind scan multiple pregnancy detection system, represented in block diagram form, according to various aspects of this disclosure.

[0021] Figure 7A This is a schematic overview of the training patterns of an untrained neural network, represented in block diagram form, according to various aspects of this disclosure.

[0022] Figure 7B This is a schematic overview of the reasoning patterns or clinical use patterns of a trained neural network, represented in block diagram form, based on various aspects of this disclosure.

[0023] Figure 8 It is a schematic diagram illustrating the examination of anatomical structures (e.g., head, heart, abdomen, pelvis, placenta, membranes, etc.) in block diagram form according to various aspects of this disclosure.

[0024] Figure 9 This is a schematic diagram illustrating an example method for detecting the intertwin separator membrane based on various aspects of this disclosure, presented in flowchart form.

[0025] Figure 10 An ultrasound image frame containing the boundary frame of the intertwin septum, according to various aspects of this disclosure, indicates that the intertwin septum has been detected.

[0026] Figure 11 This is a schematic diagram of an example method for detecting multiple coexisting fetal portions within all sweeps of an image frame, within a sweep, or in a blind scan protocol, in accordance with various aspects of this disclosure, represented in a hybrid flowchart and block diagram format.

[0027] Figure 12 The ultrasound image frame, which includes a single placental bounding box and two separate, non-overlapping head bounding boxes, indicates that two fetal heads have been detected, is based on some aspects of this disclosure.

[0028] Figure 13 This is a schematic diagram of an example fetal site geometry mapping method, represented in the form of a hybrid block diagram / flowchart, based on various aspects of this disclosure.

[0029] Figure 14 Schematic diagrams illustrating various aspects of the scanning process according to this disclosure.

[0030] Figure 15A Graphical representation of horizontal scanning or horizontal sweeping of various aspects of this disclosure.

[0031] Figure 15B It is a graphical representation of the distance calculated for horizontal sweep based on various aspects of this disclosure.

[0032] Figure 16A Graphical representation of vertical scanning or vertical sweeping of various aspects of the disclosed content.

[0033] Figure 16B It is a graphical representation of the distance calculation for vertical sweep based on various aspects of this disclosure.

[0034] Figure 17 The screen display of an ultrasound blind scan multiple pregnancy detection system is an example of various aspects of this disclosure.

[0035] Figure 18The spatial mapping screen display is an example of an ultrasound blind scan multiple pregnancy detection system based on various aspects of this disclosure.

[0036] Figure 19 It is a schematic diagram of an example fetal site geometric sequence method represented in the form of a mixed block diagram / flowchart, based on various aspects of this disclosure.

[0037] Figure 20 It is a schematic diagram representing an example fetal site geometric sequence method in the form of a block diagram, based on various aspects of this disclosure.

[0038] Figure 21A It is a schematic diagram representing an example fetal site geometric sequence method in the form of a block diagram, based on various aspects of this disclosure.

[0039] Figure 21B It is a schematic diagram representing an example fetal site geometric sequence method in the form of a block diagram, based on various aspects of this disclosure.

[0040] Figure 22 It is a schematic diagram representing an example fetal site geometric sequence method in the form of a block diagram, based on various aspects of this disclosure. Detailed Implementation

[0041] According to at least one aspect of this disclosure, an ultrasound blind-scan multiple pregnancy detection system is provided, which is capable of identifying anatomical structures of interest, such as those indicating the presence of two or more fetuses, based on a blind-scan protocol of an ultrasound imaging probe. The ultrasound blind-scan multiple pregnancy detection system presents a novel imaging quality control method that ensures the detection of features such as multiple heads or multiple spines, and uses the location of these detected features to, for example, determine whether the pregnancy is a multiple pregnancy, and thus whether specialist care may be required.

[0042] Ultrasound imaging is a vital component of high-quality obstetric care. In rural and resource-poor communities, the scarcity of ultrasound imaging results in a significant gap in maternal health care. Improving the detection of pregnancy complications through ultrasound allows for the appropriate referral of patients to more resourceful and well-trained centers for delivery care. This disclosure aims to overcome this barrier to ultrasound access in a locally sustainable and resource-efficient manner by combining standardized blind scanning protocols with artificial intelligence, thereby eliminating the need for interpreters (e.g., radiologists or obstetricians) and experienced sonographers in remote locations.

[0043] Monitoring intrauterine structures is crucial for normal fetal development and perinatal outcomes. Women with multiple pregnancies may face a higher risk of adverse maternal, infant, and postpartum outcomes. This disclosure provides a computer-aided simplified triage (CAST) setup for ultrasound systems to help novice users (e.g., minimally trained midwives) utilize algorithms for screening multiple pregnancies using obstetric ultrasound that pre-select appropriate referral cases for delivery care by community health specialists / trained healthcare professionals in more resource-rich centers. Automated CAST can help rural health providers with minimal training / lack of expertise, such as midwives, screen for benign cases (e.g., low-risk or non-urgent cases that may not require referral), avoiding additional steps in the workflow and improving the screening capacity of community medicine.

[0044] Obstetric blind scan protocols can quickly train healthcare workers to acquire high-quality ultrasound images of pregnant women. Encouraging results have been achieved with this type of blind scan protocol in determining gestational age and fetal position. Since multiple pregnancies are a type of high-risk pregnancy, it is necessary to have automated processes for detecting them.

[0045] Users can be easily trained to perform blind scans. Combining this with AI-guided automatic detection of pregnancies with more than a singleton and classifying them as multiple pregnancies would be a significant advantage in settings where high-risk pregnancies can be referred for further diagnosis and tertiary care. One objective of this disclosure is to identify multiple pregnancies based on the automatic detection of coexisting multiple anatomical structures. These can be of the same type, such as the heads of two fetuses, or they can be different anatomical structures, such as axial head and axial abdomen / heart. Multiple detections can be performed within a single frame, a single scan, or across multiple scans in a blind scan protocol. The identification of fetal membranes may be statistically significant in their frequency of occurrence, and these membranes can be re-identified by AI solutions, such as the detection module of YOLO, to support the presence of twin pregnancies.

[0046] This invention provides a method for identifying the number of fetuses in a mother's uterus at a time. Since multiple pregnancies are high-risk pregnancies that can lead to maternal and infant complications, diagnosis and referral to a higher-level center for confirmation, determination of chorionicity and amniotic susceptibility, and follow-up can be highly beneficial. While obstetric scanning protocols capture fetal images in a simplified manner and can be easily taught to users, AI-guided automated fetal detection allows users to determine the number of fetuses in the mother's uterus. Once a multiple pregnancy is detected, the user can refer the patient to a higher-level center for further management or follow-up. Diagnosing multiple pregnancies can help control maternal and infant complications (if any) and thus reduce perinatal mortality and morbidity.

[0047] Anatomical data can be acquired via a 3x3 or 5x5 blind scan protocol, covering a large portion of the mother's uterus. This disclosure provides a method for identifying such multiple pregnancies based on these scanned images (cine sequence loops or cine sequence scans). By filtering out benign cases (e.g., low-risk, non-urgent cases that may not require referral), the system avoids additional steps in the workflow, saving time and effort while freeing up experienced healthcare professionals, particularly in poorer areas with clinician shortages.

[0048] Some aspects of this disclosure may include features described in U.S. Provisional Application No. 63 / 540740, filed September 27, 2023, entitled “Ultrasound Imaging With Ultrasound Probe Guidance In Blind Sweep Protocol” and / or U.S. Provisional Application No. 63 / 540755, filed September 27, 2023, entitled “Ultrasound Imaging with Follow UpSweep Guidance After Blind Sweep Procedure”, which are incorporated herein by reference as if fully listed herein.

[0049] The ultrasound blind scan multiple pregnancy detection system provides multiple pregnancy detection methods based on the blind scan protocol.

[0050] In one aspect, frame-level anatomical coexistence can serve as a marker of multiple pregnancies. Ultrasound blind-scan multiple pregnancy detection systems identify such feature coexistences in data acquired through blind-scan protocols. These coexistences can be two fetal anatomical sites (e.g., only one of each fetus's anatomical sites), such as the heads of two fetuses, or the axial appearance of one fetal head corresponding to, for example, the presence of another fetus's abdomen or heart, and vice versa. Detecting these logical coexistences can aid in the diagnosis of multiple pregnancies.

[0051] In one aspect, detecting the intertwin septum membrane (one or more) can be used to confirm or rule out a multiple pregnancy. Since fetal portions indicating a multiple pregnancy may not always be present simultaneously in the same frame, their presence may be some distance from each other, which can be understood by examining the entire inter-scan / intra-scan dataset. The presence of the membrane (with a moderate incidence) may be a secondary diagnostic feature in such cases. Clinically, it may be highly visible and therefore can be targeted for separation using any detection module.

[0052] Previously, the identification of these relevant clinical features in order to determine or rule out multiple pregnancies had not been solved in an algorithmic sense.

[0053] Identification of multiple pregnancies depends on a number of factors. Repeated / multiple examinations of the same category of anatomical structures within the same frame are one of the key factors in identifying multiple pregnancies. Intercalating membranes can be a diagnostic factor used to distinguish between two or more fetuses. However, membranes are not always detectable or identifiable in all twin pregnancies. For example, monochorionic monoamniotic twins do not have an intercalating membrane. Technical issues, such as sampling site, ultrasound incidence angle, ultrasound settings, and machine quality, can also affect the visualization and appearance of the membranes.

[0054] Nevertheless, statistically, the incidence of fraternal twins (almost all of which are dichorionic diamniotic) is high in natural pregnancies (approximately 76% of twin pregnancies). In monochorionic twin pregnancies, about one-third are dichorionic diamniotic, and nearly two-thirds are monochorionic diamniotic. Therefore, the probability of finding the separating membrane is high. Thus, combining an anatomical structure detector for identifying duplicate anatomical structures with membrane detection can reduce false negatives. Based on training with appropriate annotations, the detector module can detect the head, heart, stomach, spine, etc. Detecting multiple heads, hearts, or stomachs in a single frame may trigger an analysis of the likelihood of a multiple pregnancy. Identifying the intertwined septum can provide additional confidence enhancement in identifying multiple pregnancies.

[0055] One aspect involves frame-level co-occurrence of anatomical structures for identifying multiple pregnancies. The co-occurrence of important fetal structures from different fetuses in a single frame image is a confirming pattern for multiple pregnancies. In clinical practice, these are often considered as confirming frames(one or more) for diagnosing a multiple pregnancy.

[0056] Algorithms for capturing coexistence: Any detection module can be used to identify different fetal parts, which in turn helps to identify the logical coexistence of multiple structures.

[0057] On the other hand, the aforementioned structural coexistences can be learned directly from B-mode images or from the spatial detection of anatomical sites (especially fetal sites) via AI algorithms. In inference mode, the system should classify scans / examinations as multiple pregnancies in real-time or near real-time and give more importance to these structural coexistences (if any).

[0058] One aspect involves targeting the intertwin septum to identify multiple pregnancies. Statistically significant: 76% of twin pregnancies are fraternal twins (diachorionic diamniotic, or dizygotic twins). Therefore, a thicker membrane is more common.

[0059] In monozygotic pregnancies (e.g., identical twins), there is approximately a 20% chance of a dichorionic diamniotic pregnancy. Therefore, the possibility of a thinner membrane also exists. The remaining twin or multiple pregnancies (accounting for about 19% of all cases) do not show any septal membrane. Therefore, if detected correctly, the presence of identifiable fetal membranes provides a convenient method for identifying the 75-80% of multiple pregnancies (MG) population.

[0060] One aspect includes the intertwin septum: these unique appearances of the intertwin septum can be identified using a detection module, which is mostly suspended in the amniotic fluid and has a statistically significant frequency of occurrence.

[0061] This disclosure significantly assists minimally trained users in capturing high-quality ultrasound-based diagnostic results through automated detection of multiple pregnancies. The ultrasound blind-scan multiple pregnancy detection system disclosed herein, implemented via a processor that communicates with the ultrasound probe, provides a substantial improvement in the quality of care available to patients in underserved areas. This improved imaging approach transforms a process that previously heavily relied on specialized experience into an accurate and repeatable process even for minimally trained personnel, without requiring the usual training of clinicians, such as emergency room staff, to identify multiple pregnancies and other prenatal conditions. This unconventional approach enhances the functionality of ultrasound imaging systems by providing reliable, repeatable imaging and diagnosis in hospital, office, vehicle, field, and home settings, and by recommending referrals for patients suspected of having multiple pregnancies.

[0062] The ultrasound blind scan multiple pregnancy detection system can be implemented as a process that is at least partially viewable on a display and operated by a control process executed on a processor that accepts user input from a keyboard, mouse, or touchscreen interface and communicates with one or more sensor probes. In this respect, the control process performs specific operations in response to different inputs or selections made at different times. The specific structure, function, and operation of the processor, display, sensors, and user input system provide novel features or aspects of this disclosure.

[0063] These descriptions are provided for illustrative purposes only and should not be construed as limiting the scope of ultrasound blind scan multiple pregnancy detection systems. Specific features may be added, deleted, or changed without departing from the spirit of the subject matter for which protection is sought.

[0064] For the purpose of facilitating an understanding of the principles of this disclosure, reference will now be made to the aspects illustrated in the accompanying drawings, and they will be described using specific language. However, it should be understood that this disclosure is not intended to limit the scope of the disclosure. Any modifications and further alterations to the described devices, systems, and methods, as well as any further applications of the principles of this disclosure, are fully contemplated and included in this disclosure, as would normally occur to those skilled in the art. In particular, it is fully contemplated that features, components, and / or steps described with respect to one aspect can be combined with features, components, and / or steps described with respect to other aspects of this disclosure. However, for the sake of brevity, numerous iterations of these combinations will not be described separately.

[0065] Figure 1 This is a schematic diagram of an ultrasound imaging system 100 according to aspects of this disclosure. For example, the ultrasound imaging system 100 can be used to acquire ultrasound video scans, which can then be analyzed by a human clinician or artificial intelligence to diagnose medical conditions.

[0066] An ultrasound imaging system 100 is used to scan a region or volume of a subject's body. The subject may include a patient undergoing the ultrasound imaging procedure, or any other person, or any suitable biological or non-biological body or structure. The ultrasound imaging system 100 includes an ultrasound imaging probe 110, which communicates with a host computer 130 via a communication interface or link 120. The probe 110 may include a transducer array 112, a beamformer 114, processor circuitry 116, and a communication interface 118. The host computer 130 may include a display 132, processor circuitry 134, a communication interface 136, and a memory 138 for storing subject information.

[0067] In some aspects, probe 110 is an external ultrasound imaging device comprising a housing 111 configured for handheld operation by a user. A transducer array 112 may be configured to acquire ultrasound data when the user holds the housing 111 of probe 110, such that the transducer array 112 is located near or in contact with the skin of the subject. Probe 110 is configured to acquire ultrasound data of internal anatomical structures of the subject when probe 110 is located outside the subject. In some aspects, probe 110 may be an external ultrasound probe, a transthoracic probe, and / or a curved array probe.

[0068] In other aspects, probe 110 may be an internal ultrasound imaging device and may include a housing 111 configured to be placed within the body of an object. In some aspects, probe 110 may be a curved array probe. Probe 110 may be any suitable form for any suitable ultrasound imaging application, including external and internal ultrasound imaging.

[0069] For an ultrasound imaging device, transducer array 112 transmits ultrasound signals toward an anatomical target 105 of the object and receives echo signals reflected back to transducer array 112 from the target 105. Ultrasonic transducer array 112 may include any suitable number of acoustic elements, including one or more acoustic elements and / or multiple acoustic elements. In some instances, transducer array 112 includes a single acoustic element. In some instances, transducer array 112 may include an array of acoustic elements having any number of acoustic elements in any suitable configuration. For example, transducer array 112 may contain from 1 acoustic element to 10,000 acoustic elements, including 2 acoustic elements, 4 acoustic elements, 36 acoustic elements, 64 acoustic elements, 128 acoustic elements, 500 acoustic elements, 812 acoustic elements, 1,000 acoustic elements, 3,000 acoustic elements, 8,000 acoustic elements, etc., and / or other values ​​larger or smaller. In some instances, transducer array 112 may comprise an array of acoustic elements having any number of acoustic elements, which may be in any suitable configuration, such as linear arrays, planar arrays, curved surface arrays, spherical arrays, ring arrays, phased arrays, matrix arrays, one-dimensional (1D) arrays, 1.x-dimensional arrays (e.g., 1.5D arrays), or two-dimensional (2D) arrays. The array of acoustic elements (e.g., one or more rows, one or more columns, and / or one or more orientations) may be controlled and activated uniformly or independently. Transducer array 112 may be configured to obtain one-dimensional, two-dimensional, and / or three-dimensional images of an object's anatomy. In some aspects, transducer array 112 may comprise a piezoelectric micromechanical ultrasonic transducer (PMUT), a capacitive micromechanical ultrasonic transducer (CMUT), a single crystal, lead zirconate titanate (PZT), PZT composites, other suitable transducer types, and / or combinations thereof.

[0070] The object 105 may include any anatomical structure or feature, such as the abdomen of a pregnant patient, one or more fetuses in the abdomen of a pregnant patient, etc.

[0071] Beamformer 114 is coupled to transducer array 112. Beamformer 114 controls transducer array 112, for example, for transmitting ultrasound signals and receiving ultrasound echo signals. In some aspects, beamformer 114 may apply a time delay to signals transmitted to individual acoustic transducers within the array of transducers 112, causing the acoustic signals to be redirected to propagate in any suitable direction away from probe 110. Based on the response of the received ultrasound echo signals, beamformer 114 further provides image signals to processor circuitry 116. Beamformer 114 may include multiple stages of beamforming. Beamforming can reduce the number of signal lines coupled to processor circuitry 116. In some aspects, the combination of transducer array 112 and beamformer 114 may be referred to as an ultrasound imaging component.

[0072] Processor 116 is coupled to beamformer 114. Processor 116 may also be described as processor circuitry, which may include other components communicating with processor 116, such as memory, beamformer 114, communication interface 118, and / or other suitable components. Processor 116 may include a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), controller, field-programmable gate array (FPGA) device, another hardware device, firmware device, or any combination thereof configured to perform the operations described herein. Processor 116 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Processor 116 is configured to process beamformed image signals. For example, processor 116 may perform filtering and / or quadrature demodulation to modulate the image signals. Processor 116 and / or 134 may be configured to control array 112 to obtain ultrasound data associated with object 105.

[0073] Communication interface 118 is coupled to processor 116. Communication interface 118 may include one or more transmitters, one or more receivers, one or more transceivers, and / or circuitry for transmitting and / or receiving communication signals. Communication interface 118 may include hardware and / or software components implementing a specific communication protocol suitable for transmitting signals to host 130 via communication link 120. Communication interface 118 may be referred to as a communication device or communication interface module.

[0074] Communication link 120 can be any suitable communication link. For example, communication link 120 can be a wired link, such as a Universal Serial Bus (USB) link or an Ethernet link. Alternatively, communication link 120 can be a wireless link, such as an Ultra Wideband (UWB) link, an IEEE 802.11 WiFi link, or a Bluetooth link.

[0075] At host 130, communication interface 136 can receive image signals. Communication interface 136 can be substantially similar to communication interface 118. Host 130 can be any suitable computing and display device, such as a workstation, personal computer (PC), laptop computer, tablet computer, or mobile phone.

[0076] Processor 134 is coupled to communication interface 136. Processor 134 may also be described as processor circuitry, which may include other components communicating with processor 134, such as memory 138, communication interface 136, optional speaker 139, and / or other suitable components. Processor 134 may be implemented using a combination of software and hardware components. Processor 134 may include a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), controller, FPGA device, other hardware device, firmware device, or any combination thereof configured to perform the operations described herein. Processor 134 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Processor 134 may be configured to generate image data based on image signals received from probe 110. Processor 134 may apply advanced signal processing and / or image processing techniques to the image signals. In some aspects, processor 134 may form a three-dimensional (3D) volumetric image based on the image data. In some aspects, processor 134 can perform real-time processing on image data to provide an ultrasound image stream video of target 105. In some aspects, host 130 includes a beamformer. For example, processor 134 may be part of and / or otherwise communicate with such a beamformer. The beamformer in host 130 may be a system beamformer or a master beamformer (providing one or more subsequent beamforming stages), while beamformer 114 may be a probe beamformer or a microwave beamformer (providing one or more initial beamforming stages).

[0077] Memory 138 is coupled to processor 134. Memory 138 can be any suitable storage device, such as cache memory (e.g., cache memory of processor 134), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state memory device, hard disk drive, solid-state drive, other forms of volatile and non-volatile memory, or a combination of different types of memory.

[0078] Memory 138 may be configured to store object information, measurements, data, or files relating to the object's medical history, history of surgeries performed, anatomical or biological features, characteristics, or medical conditions associated with the object, computer-readable instructions such as code, software, or other applications, and any other suitable information or data. Memory 138 may be located within host 130. Object information may include measurements, data, files, other forms of medical history, such as, but not limited to, ultrasound images, ultrasound videos, and / or any imaging information relating to the object's anatomy. Object information may include parameters relating to the imaging process, such as the anatomical scanning window, probe orientation, and / or the object's position during the imaging process. Memory 138 may also be configured to store information relating to the training and implementation of machine learning algorithms (e.g., neural networks), and / or information relating to the implementation of image recognition algorithms, image quantization algorithms, and / or image acquisition guidance algorithms for detecting / segmenting anatomical structures, including those described herein.

[0079] Display 132 is coupled to processor circuitry 134. Display 132 may be a monitor or any suitable display. Display 132 is configured to display ultrasound images, video images, and / or any imaging information of target 105.

[0080] Ultrasound imaging system 100 can be used to assist an ultrasound physician in performing ultrasound scans. Scans can be performed in a point-of-care environment. In some instances, host 130 is a console or mobile cart. In some instances, host 130 can be a mobile device, such as a tablet, mobile phone, or laptop. During the imaging process, the ultrasound system can acquire ultrasound images of specific regions of interest within the anatomical structures of the subject. Ultrasound imaging system 100 can then analyze the ultrasound images to identify various parameters associated with the image acquisition, such as scan window, probe orientation, subject position, and / or other parameters. Ultrasound imaging system 100 can then store the images and these associated parameters in memory 138. In subsequent imaging processes, ultrasound imaging system 100 can retrieve previously acquired ultrasound images and associated parameters for display to the user, which can be used to guide the user of ultrasound imaging system 100 in using the same or similar parameters in subsequent imaging processes, as described in more detail below.

[0081] In some aspects, processor 134 may utilize a deep learning-based predictive network to identify parameters of the ultrasound image, including anatomical scanning window, probe orientation, object position, identification and location of anatomical features, and / or other parameters. In some aspects, processor 134 may receive measurements related to the physiological state of the region of interest or object being imaged during the imaging process or perform various calculations related to the physiological state of the region of interest or object being imaged. These measurements and / or calculation results may also be displayed to the sonographer or other user via display screen 132.

[0082] In some respects, the host unit 130 may also include a speaker 180. For example, the speaker 180 may be used to provide the user with alert sounds, beeps, or other auditory feedback.

[0083] Before proceeding, it should be noted that the examples described above are for illustrative purposes only and are not intended to be limiting. Other devices and / or device configurations can be used to perform the operations described herein.

[0084] Figure 2 This is a schematic diagram of processor circuitry 250 according to aspects of this disclosure. Processor circuitry 250 can be implemented in ultrasound imaging system 100, or in other devices or workstations (e.g., third-party workstations, network routers, etc.), or on a cloud processor or other remote processing unit to implement the method as needed. As shown, processor circuitry 250 may include processor 260, memory 264, and communication module 268. These components can communicate directly or indirectly with each other, for example, via one or more buses.

[0085] Processor 260 may include a central processing unit (CPU), a digital signal processor (DSP), a controller, or any combination of general-purpose computing devices, reduced instruction set computing (RISC) devices, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other related logic devices, including mechanical computers and quantum computers. Processor 260 may also include other hardware devices, firmware devices, or any combination thereof configured to perform the operations described herein. Processor 260 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration.

[0086] Memory 264 may include cache memory (e.g., the cache memory of processor 260), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state storage devices, hard disk drives, other forms of volatile and non-volatile memory, or combinations of different types of memory. In one aspect, memory 264 includes a non-transitory computer-readable medium. Memory 264 may store instructions 266. Instructions 266 may include instructions that, when executed by processor 260, cause processor 260 to perform the operations described herein. Instructions 266 may also be referred to as code. The terms “instruction” and “code” should be interpreted broadly to include any type of computer-readable statement(s). For example, the terms “instruction” and “code” may refer to one or more programs, routines, subroutines, functions, procedures, etc. “Instruction” and “code” may include a single computer-readable statement or multiple computer-readable statements.

[0087] The communication module 268 may contain any electronic circuitry and / or logic circuitry to facilitate direct or indirect data communication between the processor circuitry 250 and other processors or devices. In this regard, the communication module 268 may be an input / output (I / O) device. In some instances, the communication module 268 facilitates direct or indirect communication between the processor circuitry 250 and / or the various components of the ultrasound imaging system 100. The communication module 268 can communicate within the processor circuitry 250 using various methods or protocols. Serial communication protocols may include, but are not limited to, the US Serial Protocol Interface (US SPI), Inter-Integrated Circuit Protocol (IIP), and other protocols. 2C) Recommended standards RS-232, RS-485, Controller Area Network (CAN), Ethernet, ARINC 429, MODBUS, MIL-STD-1553, or any other suitable method or protocol. Parallel protocols include, but are not limited to, Industry Standard Architecture (ISA), Advanced Technology Attachment (ATA), Small Computer System Interface (SCSI), Peripheral Component Interconnect (PCI), IEEE 488, IEEE 1284, and other suitable protocols. Where appropriate, serial and parallel communications can be bridged using a Universal Asynchronous Receiver / Transmitter (UART), a Universal Synchronous Receiver / Transmitter (USART), or other suitable subsystems.

[0088] External communication (including but not limited to software updates, firmware updates, model sharing between the processor and the central server, or readings from the ultrasound imaging system 100) can be implemented using any suitable wireless or wired communication technology, such as cable interfaces (e.g., Universal Serial Bus (USB), Micro USB, Lightning, or FireWire interfaces), Bluetooth, Wi-Fi, ZigBee, Li-Fi, or cellular data connections (e.g., 2G / GSM, 3G / UMTS, 4G, LTE, WiMax, or 5G). For example, a Bluetooth Low Energy (BLE) radio module can be used to establish connections with cloud services, transmit data, and receive software patches. The controller can be configured to communicate with remote servers or local devices (e.g., laptops, tablets, or handheld devices), or may include a display capable of showing status variables and other information. Information can also be transmitted via physical media, such as USB flash drives or memory sticks.

[0089] Figure 3A This is a set of schematic diagrams representing different types of twins, 200, 201, surrounded by at least one amniotic sac or amnion 210 and at least one chorionic sac or chorion 315 within the patient's uterus, according to various aspects of this disclosure. At least one placenta 240 is also visible. Automated detection of multiple pregnancies or multiple gestational pregnancies (e.g., twins) is the objective of this disclosure.

[0090] In the first example 310, the multiple pregnancy is monochorionic (e.g., only one chorion 315) and monoamniotic (e.g., only one amniotic sac 210 containing two fetuses 200, 201 and having a single placenta 240).

[0091] In the second example 320, the multiple pregnancy is monochorionic (e.g., only one chorion 315), but dipromniform (e.g., two amniotic sacs 210, 211, separated by fetal membranes 350, each amniotic sac containing one fetus 200, 201), and has a single placenta 240. Detection of the fetal membranes is a method for detecting multiple pregnancies or multiple gestational pregnancies, as described below.

[0092] In the third example 330, the multiple pregnancy is a diporaminal pregnancy (e.g., having two chorions 315, 316 separated by their respective membranes 350) and a dipaminal pregnancy (e.g., having two amniotic sacs 210, 211, each amniotic sac containing one of the twins 200, 201), and two placentas 240 fused together.

[0093] In the fourth example 340, the multiple pregnancy is a diporaminal (e.g., having two separate chorions 315, 316, separated by their respective membranes 350) and a dipaminal (e.g., having two separate amniotic sacs 200, 201, each amniotic sac containing one of the twins 200, 201), and two separate placentas 240, 241. This type of twin pregnancy has a lower incidence of complications compared to monochorionic or monoamniotic twin pregnancies.

[0094] 70% of twin pregnancies are fraternal twins (e.g., formed from two fertilized eggs), and fraternal twin pregnancies are considered to be dichorionic and diamniotic pregnancies. 30% of pregnancies are monozygotic pregnancies (e.g., developing from a single fertilized egg), and 20% of these are also dichorionic and diamniotic pregnancies. Therefore, detecting the membranes separating the two amniotic sacs at 35° can effectively detect up to 76% of twin pregnancies. The remaining 24% of detection may rely on other characteristics of the pregnancy, such as the repeated appearance of unique anatomical features (e.g., two heads, two hearts, two pelvises, etc.) and / or the spatial / geometric relationships between fetal parts, as described below.

[0095] The uterus can be viewed as a maternal anatomical structure. Depending on the context, the placenta can be viewed as a fetal anatomical structure, a maternal anatomical structure, or an interface between the two.

[0096] Figure 3B This is a set of schematic cross-sectional views of two fetuses 200, 201 arranged in different positions within a patient's uterus, as observed via an ultrasound imaging plane 360, according to various aspects of this disclosure. The imaging plane 360 ​​is an example of an imaging plane (e.g., a frame in a single scan), and the same imaging plane is displayed in all fetal positions of the twins, indicating that different fetal portions can be seen in the same imaging plane depending on the fetal position of the twins.

[0097] In the first example 370, twins 200 and 201 are both in a cephalic presentation (head down). Approximately 45% of twin pregnancies fall into this category. In example 370, the imaging plane 360 ​​may not contain any duplicate anatomical structures.

[0098] In the second example 375, one twin 200 is in breech (head up) and the other is in cephalic (head down). Approximately 37% of twin pregnancies fall into this category. Breech delivery is associated with a higher rate of complications. In example 375, the imaging plane 360 ​​includes the heads of both fetuses 200 and 201.

[0099] In example 380, twins 200 and 201 are both in breech presentation (head up). Approximately 10% of twin pregnancies fall into this category. In example 380, the imaging plane 360 ​​may not contain any duplicate anatomical structures.

[0100] In Example 385, one twin 200 is in a transverse (lateral) position, and the other twin 201 is in a cephalic presentation. Approximately 5% of twin pregnancies fall into this category. Cephalic presentation delivery is associated with a higher rate of complications. In Example 385, the imaging plane 360 ​​may include the hearts of both fetuses 200 and 201.

[0101] In example 390, one twin 200 is in a breech (head up) position, and the other twin 201 is in a transverse (lateral) position. Approximately 2% of twin pregnancies fall into this category. In example 390, the imaging plane 360 ​​may include the hearts of both fetuses 200 and 201.

[0102] In Example 395, twins 200 and 201 are both in the head-down position. Approximately 0.5% of twin pregnancies fall into this category. In Example 395, the imaging plane 360 ​​may include the hearts of both fetuses 200 and 201.

[0103] Figure 4 This is a schematic diagram of a patient 300 whose abdomen follows a blind ultrasound scan protocol, according to aspects of this disclosure. Visible on the abdomen 310 of the patient 300 is a desired scan pattern 320, designed to capture images of desired features of the patient's anatomical structures. The scan pattern 320 includes multiple vertical scan lines 330 and multiple horizontal scan lines 340. Each scan line 330, 340 represents the desired path of a single imaging scan of the abdomen 310. Figure 4In the example shown, the sweep pattern includes three vertical sweep lines 330, labeled L (patient's left), M (patient's middle), and R (patient's right), all upward relative to the patient; and three horizontal sweep lines 340, labeled C1 (bottom), C2 (middle), and C3 (top), all from right to left relative to the patient. However, it should be understood that sweep pattern 320 may include more or fewer sweep lines 330, including vertical sweep lines 330, horizontal sweep lines 330, or combinations thereof, based on any combination of upward, downward, leftward, or rightward directions according to the patient's fundal height. For example, if the patient has a large abdomen, more sweeps may be required. Furthermore, sweep pattern 320 may cover other parts of the patient's body, including but not limited to the head, neck, spine, limbs, etc. Types of blind sweep protocols include, but are not limited to, obstetric sweep imaging (OSI), volume sweep imaging (VSI), 6-stage, gestational age machine learning project (FAMLI), and Philips.

[0104] These sweep patterns represent the desired probe motion information when the ultrasound probe acquires multiple ultrasound image frames during sweeping, including the desired position, desired velocity, and / or desired ultrasound probe orientation. It is noteworthy that the desired sweep pattern or blind scan protocol stored in the processor memory may contain only vertical sweeps, only horizontal sweeps, or grids that may contain both vertical and horizontal sweeps (e.g., 3x3, 5x5, etc.), and may contain associated parameters stored in memory, such as the desired probe motion (e.g., position, velocity, and / or orientation) (e.g., ...). Figure 5 The blind scan protocol (440) is relevant. Depending on the implementation, sweeps can include curve sweeps, diagonal sweeps, and other types of sweeps. For example, the protocol and its associated parameters can be based on standards developed by authoritative bodies in the field (physician organizations, ultrasound physician organizations, etc.), which are published in academic journals / textbooks, etc.

[0105] Figure 5 This is a schematic diagram of an example ultrasound blind scan multiple pregnancy detection system 400, represented in a hybrid block diagram / flowchart format, according to various aspects of this disclosure. An ultrasound probe 110, operated by a novice user 410, performs a blind scan protocol 440 on the patient 300's body and sends ultrasound imaging data to a host device 130, such as a tablet, smartphone, ultrasound cart, etc. In step 415, the host device 130 uses the ultrasound image data acquired by the ultrasound probe 110 to generate an ultrasound image. The host device 130 can control the ultrasound probe 110 to acquire ultrasound image data (e.g., the host device 130 establishes communication with the ultrasound probe 110, the host device 130 sends control signals to start and / or stop acquiring ultrasound image data, the host device 130 sends power signals to power the ultrasound probe 110, etc.).

[0106] In step 420, the host computer 130 uses the ultrasound image generated in step 415 and / or the ultrasound image data used to generate the ultrasound image to determine whether the pregnant woman has a multiple pregnancy, as will be described in more detail below. In step 430, the host computer generates a visual representation on the display screen showing an indication of the presence of a multiple pregnancy and / or graphics related to the determination of a multiple pregnancy.

[0107] In step 450, host 130 determines whether the planned path of blind scan protocol 440 has been adequately followed. Otherwise, execution proceeds to steps 430 and 460, where host 130 provides feedback (e.g., audio feedback via a speaker and / or visual feedback from a display) to repeat one or more scans of blind scan protocol 440. In some aspects, the determination in step 450 may be based on the results of the analysis in step 420 regarding whether the patient has a multiple pregnancy. For example, if the ultrasound image data is insufficient to perform the determination in step 420, or if the obtained ultrasound image data is unsuitable for performing the determination in step 420, then step 450 may determine that one or more scans of blind scan protocol 440 were not correctly completed. In steps 430 and 460, the user may be informed of the need for one or more scans to be repeated via visual or audio feedback.

[0108] In some aspects, the planned path of the blind scan protocol 440 determined in step 450 can be used by the host 130 in step 420 to determine whether the patient has a multiple pregnancy. For example, the planned path determined in step 450 can provide absolute or relative spatial information (e.g., vertical position, horizontal position) regarding the ultrasound image data acquired by the ultrasound probe and / or the ultrasound images generated by the host. For example, referring again... Figure 4 The planned path determined in step 450 can indicate that M sweeps in the horizontal direction ( Figure 4 (In the left-right direction) it is located between the R-scan and L-scan. The host 130 can use the spatial information in step 450 to process ultrasound image data and / or ultrasound images to determine in step 420 whether a multiple pregnancy exists.

[0109] Based on visual representation 430 and / or audio guidance 460, the novice user 410 makes a triage decision to rule out or confirm a multiple pregnancy. In step 470, the novice user 410 rules out a multiple pregnancy; and in step 480, the novice user 410 performs or recommends routine periodic assessments for the patient 300. In step 490, the novice user 410 confirms a multiple pregnancy; and in step 495, the novice user 410 refers the patient to a specialist user, such as an experienced sonographer, radiologist, obstetrician, or other physician, for further imaging and diagnosis.

[0110] The flowcharts and block diagrams provided herein are for illustrative purposes; numerous variations will be apparent to those skilled in the art, but these variations still fall within the scope of this disclosure. For example, any step described herein may optionally output information related to that step to the user, thereby providing information that is not otherwise available, and thus representing an improvement over the prior art in terms of user interface. Similarly, block diagrams may illustrate a particular arrangement of components, modules, services, steps, processes, or layers, resulting in a particular data flow. It should be understood that some embodiments of the systems disclosed herein may include other components, some embodiments may lack some of the components shown, and the arrangement of components may differ from that shown, resulting in different data flows, while still performing the methods described herein. The logic of the flowcharts may be shown as sequential. However, similar logic may be parallel, massively parallel, object-oriented, real-time, event-driven, cellular automata-like, or other forms, while implementing the same or similar functionality. To perform the methods described herein, a processor may break down each step described herein into multiple machine instructions, and may execute these instructions on a single processor or across multiple processors at a rate of hundreds, thousands, millions, or billions of instructions per second. Such rapid execution may be necessary to perform the method in real-time or near real-time as described herein. For example, to assess fetal anatomical location and / or indicate whether it is a multiple pregnancy, the system may need to generate a visual representation within one second of completing the blind scan protocol.

[0111] Figure 6This is a schematic diagrammatic representation, in block diagram form, of at least a portion of an example ultrasound blind scan multiple pregnancy detection system 400 according to various aspects of this disclosure. A sequence of ultrasound images 610, also known as a cinema loop or cinema sequence scan (e.g., one cinema sequence scan per sweep of a blind scan protocol), is received by an object detector 620 (whether received one at a time, simultaneously, or in groups). Each cinema sequence scan may contain multiple ultrasound image frames (e.g., 100-1000 ultrasound image frames, and / or other larger or smaller values). For example, the object detector 620 may be a machine learning (ML) neural network as described below, but other types of object detectors, including classical image recognition algorithms, may be used instead or in addition. The object detector 620 may be a deep learning network (e.g., a convolutional neural network or a CNN) trained to detect maternal and / or fetal anatomy. For example, the output of the object detector may include an annotated version of the cinema loop 610 containing bounding boxes for each detected anatomy in each frame in which anatomy was detected. The object detector 620 can generate, as its output, the detection of the intertwin septum in one or more cine sequence ultrasound image frames, and / or the detection of one or more fetal anatomical sites in one or more cine sequence ultrasound image frames. For example, the detected anatomical structures may include the fetal head, abdomen, heart, or pelvis—each fetus is expected to have only one unique feature—as well as the intertwin septum 350 and / or placenta 240 (see [link to relevant documentation]). Figure 3A ).

[0112] The output of object detector 620 is then passed to multiple pregnancy determiner 630, which may be, for example, software, hardware, firmware, analog / digital logic, analog / digital circuitry, or a combination thereof. Multiple pregnancy determiner 630 determines whether the pregnancy is a multiple pregnancy by detecting the intertwin septum and / or multiple fetal anatomical sites (from object detector 620). Figure 6 In the example shown, the multiple pregnancy detector 630 includes a detection 900 of the intertwin septum membrane, a detection 1100 for detecting the coexistence of fetal portions within an image frame, a geometric mapping 1300 of the fetal portion, and a geometric sequence 1900 of the fetal portion. In the example, the detector 900, detection 1100, mapping 1300, and sequence 1900 can have a simple binary output indicating whether a multiple pregnancy has been detected. This detection can occur individually in each frame of each movie loop 610, in all frames of the movie loop 610, or together in all movie loops.

[0113] Optionally, combiner 640 can aggregate the outputs of determinations 900, 1100, 1300, and 1900. Combiner 640 can be software, hardware, firmware, analog / digital logic, analog / digital circuitry, or a combination thereof. For example, combiner 640 can use a majority voting system to determine whether a multiple pregnancy has been detected, such that three out of four determinations must be in favor for a multiple pregnancy to be determined. In other aspects, if any one of determinations 900, 1100, 1300, or 1900 detects a multiple pregnancy, combiner 640 can indicate a multiple pregnancy. In still other aspects, combiner 640 may need to determine unanimous agreement among determinations 900, 1100, 1300, or 1900. In still other aspects, if any two of determinations 900, 1100, 1300, or 1900 detect a multiple pregnancy, the combiner can indicate a multiple pregnancy. In some aspects, the combiner 640 can assign different weights to the outputs determining 900, 1100, 1300, or 1900. For example, the "yes" output for determining 900, 1100, 1300, or 1900 might be assigned a value of 1, and the "no" output for determining 900, 1100, 1300, or 1900 might be assigned a value of 0. The combiner 640 might require a total value of 2 to output a multiple pregnancy detection result. For example, if the output for determining 900 is weighted by a weight factor of 1.5, then the "yes" value of the output for determining 900 provides a value of 1.5 (1x1.5) within the required total value of 2. In still other aspects, the combiner can be a trained machine learning network that takes the outputs determining 900, 100, 1300, and 1900 as input and generates a yes / no output indicating whether a multiple pregnancy is suspected. In some aspects, the combiner 640 is omitted.

[0114] Processor (e.g., Figure 1 The processor 138 in Figure 1 The processor 116 and / or other processors in the system communicate with the display (e.g., Figure 1 The display 132 and / or other displays provide output indicating whether the pregnancy is a multiple pregnancy. For example, the output of the combiner 640, the output of the multiple pregnancy determiner 630, and / or the output of individual determinations 900, 1100, 1300, or 1900 are presented in the form of a visual representation 430 (e.g., Figure 1The visual representation 430 can serve as a binary indicator indicating whether a multiple pregnancy has been detected. The visual representation 430 may include an indication 650 (e.g., text or a symbol) indicating whether a multiple pregnancy is suspected. The screen display 430 may also include a graphical representation 660 associated with determining whether a multiple pregnancy is suspected. For example, the graphical representation 660 may include a drawing, ultrasound image frame, movie loop, or generated graph, with or without text or symbols as annotations, including the graphics, images, text, and / or other visual representations described herein (e.g., the output of combiner 640, the output of multiple pregnancy determiner 630, and / or the output of individual determinations 900, 1100, 1300, or 1900).

[0115] Figure 7A This is a schematic overview, in block diagram form, of the training mode 700 of an untrained neural network 710a, according to various aspects of this disclosure. Figure 7A In the example shown, a set of training data 705a includes ultrasound cine loops swept by the probe, with corresponding anatomical structures (head, heart, abdomen, pelvis, membranes, placenta, etc.) labeled. The training data 705a is fed into an untrained neural network 710a during iterative training, as is familiar to those skilled in the art.

[0116] The network model's parameters (e.g., the weights of each artificial neuron) are initialized with initial values ​​A, which can be random values ​​or results from training on previous datasets. During iteration, the network is used to perform detection inference on the training images, compare the results with baseline ground truth labels, and use an optimizer to tune the network parameters B until the accuracy metric is maximized.

[0117] Therefore, the output of the training process 700 is a trained neural network 710b, in which parameters B (e.g., weights) are optimized to generate accurate bounding boxes for the anatomical structures imaged in the training data 705a.

[0118] Figure 7B This is a schematic overview, in block diagram form, of the inference mode or clinical use mode 704 of a trained neural network 710b, representing various aspects of this disclosure. In clinical applications, blind-scan ultrasound video, cine loop, or cine sweep 720 is input into the trained and validated neural network 710b for analysis. The trained and validated neural network 710b then outputs anatomical detection bounding boxes 740 for each image (or the entire sweep). In some aspects, confidence values ​​can be determined as normalized values ​​in the range [0-1], where 0 indicates the lowest confidence and 1 indicates the highest confidence that the detection result is correct.

[0119] Figure 8 This is a schematic diagram illustrating the detection of anatomical structures (e.g., head, heart, abdomen, pelvis, placenta, membranes, etc.) in block diagram form according to various aspects of this disclosure. A movie loop 810 comprising multiple frames 820 is input into a trained object detector 830.

[0120] The object detector 830 can implement or include any suitable type of learning network. For example, in some aspects, the object detector 830 can include a neural network, such as a convolutional neural network (CNN). Furthermore, the CNN can additionally or alternatively be an encoder-decoder type network, or it can utilize a backbone architecture based on other types of neural networks, such as object detection networks, classification networks, etc. An example backbone network is the Darknet YOLO backbone network (e.g., Yolov3), which can be used for object detection. For example, a CNN can contain a set of N convolutional layers, where N can be any positive integer. When a CNN is the backbone network, fully connected layers can be omitted. CNNs can also contain max-pooling layers and / or activation layers. Each convolutional layer can contain a set of filters configured to extract features from the input (e.g., frames from an ultrasound video). The value N and the size of the filters may vary depending on various aspects. In some instances, the convolutional layers can use any non-linear activation function, such as leak-corrected non-linear (ReLU) activation function and / or batch normalization. Max pooling layers gradually reduce the high-dimensional output to the dimension of the desired result (e.g., bounding boxes of detected features). The output of a detection network may contain a large number of bounding boxes, most of which have very low confidence scores and are therefore filtered out or ignored. Fully connected layers can be called perceptual layers or receptive layers. In some aspects, perceptual / receptive layers and / or fully connected layers may exist in object detectors (e.g., multilayer perceptrons).

[0121] These descriptions are included for illustrative purposes; those skilled in the art will understand that other types of learning models having similar or different features as described above may be used instead of or supplementary to these descriptions without departing from the spirit of this disclosure.

[0122] The output of the object detector 830 may include an annotated movie loop 840 consisting of multiple annotated image frames 842, and may include frame-by-frame metrics 845, such as the confidence of the detection.

[0123] The systems and methods disclosed in this paper are broadly applicable to different types of features and can, for example, draw bounding boxes around the head, heart, placenta, or other anatomical features, depending on the specific implementation. Object detectors can be single-class or multi-class, depending on how the model is constructed. If another detector is trained separately, the two models can be run independently (e.g., one model for each feature type). Otherwise, multiple feature classes can be identified simultaneously and bound in detection boxes. In the example, the machine learning model for placenta detection can use the exact same structure as the model for heart detection. A single detector capable of detecting multiple feature types (multi-class detector) can be trained / run, providing their locations as outputs, along with a confidence score and feature type (category) for each detection. Alternatively, multiple single-class detectors can be run, each trained to detect a single feature type / category. These separate single-class detectors may have the same architecture (e.g., layers and connections) but are trained using different data (e.g., different images and / or annotations) and therefore have different weights.

[0124] Figure 9 This is a schematic diagram illustrating, in flowchart form, an example method 900 for detecting the intertwined membrane, according to various aspects of this disclosure. It should be understood that, in other respects, the steps of method 900 may be consistent with... Figure 9 The different sequences of execution shown may provide additional steps before, during, and after the steps, and / or may replace or eliminate some of the described steps. One or more steps of method 900 may be performed by one or more devices and / or systems described herein, such as components of ultrasound imaging system 100, ultrasound blind scan multiple pregnancy detection system 400, and / or processor circuitry 250.

[0125] In step 920, method 900 includes receiving an ultrasound image frame 910 and using an object detector (e.g., a machine learning-based object detector as described above) to detect the intertwin septum. The process then proceeds to steps 930 and 940.

[0126] In step 930, method 900 includes determining that no intertwin septum is detected in the image frame. Execution then returns to step 910 until all image frames have been analyzed, and then proceeds to step 950.

[0127] In step 940, method 900 includes drawing a bounding box of the intertwin diaphragm on the image frame, indicating the most likely boundary of the intertwin diaphragm detected. Execution then returns to step 910 until all image frames have been analyzed, and then proceeds to step 950.

[0128] In step 950, method 900 includes determining the number or percentage of frames containing the detection of the intertwin spacer membrane from multiple scans of the blind scan protocol. The process then proceeds to step 960.

[0129] In step 960, method 900 includes determining whether the quantity or percentage in step 950 exceeds a threshold 970. If yes, proceed to step 980. If no, proceed to step 990.

[0130] In step 980, method 900 includes determining a suspected multiple pregnancy. Method 900 is now complete.

[0131] In step 990, method 900 includes determining whether the possibility of a multiple pregnancy is ruled out or whether there is no suspicion of a multiple pregnancy. Method 900 is now complete.

[0132] Figure 10 An ultrasound image frame 1000, including the boundary frame 1010 of the intertwin septum membrane 350, as described in various aspects of this disclosure, indicates that the intertwin septum membrane 350 has been detected. The two fetal heads 1030 and two placentas 240 on either side of the membrane 350 are also visible. As stated above... Figure 9 As described, the detection of an intertwin septum 350 (e.g., in frames exceeding a threshold) indicates a twin pregnancy. The presence of two fetal heads is also an indicator of a twin pregnancy, as described below.

[0133] Figure 11 This is a schematic diagram of an example method 1100 for detecting multiple coexisting fetal portions within all sweeps of an image frame, within a sweep, or in a blind scan protocol, in accordance with various aspects of this disclosure, represented in a hybrid flowchart and block diagram format.

[0134] In step 1110, method 1100 includes receiving ultrasound image frames 1105 and detecting fetal body parts using an object detector (e.g., a trained neural network as described above, but other types of detectors may also be used instead or in addition to the existing ones). Execution then proceeds to steps 1115 and 1120.

[0135] In step 1115, method 1100 includes placing bounding boxes around all detected fetal body parts. The process then proceeds to steps 1125 and / or 1130.

[0136] In step 1120, method 1100 includes determining that no fetal body parts are detected in the image. The process then proceeds to step 1140.

[0137] In step 1125, method 1100 includes using a statistical model (e.g., Naive Bayes, One-Class SVM, Gaussian Mixture Model (GMM), etc.) to analyze image frames and bounding boxes and determine the statistical probability of finding the detected anatomical structure in a single frame for a single fetus.

[0138] An example implementation of the statistical model is as follows: For each frame, construct a numerical vector V of size 1xN [N = number of categories], where the value of vector V at index "idx" is the number of non-overlapping bounding boxes of the category corresponding to "idx" detected in that frame by means of 1110. Train the GMM to learn the distribution of these vectors from single-feed data. During inference / runtime, construct a similar vector for each frame and feed it into the trained GMM model, which will return the probability that the frame comes from a single-feed check. Use this probability to determine whether the frame indicates MG. Then repeat this for all frames.

[0139] In the example, if two coexisting structures are distinctive enough to be suspected of being MG, they can be separated via step 1130 (rule-based). However, a classifier can replace or supplement the rule-based approach. This classifier can learn on its own the coexistence of these defined structures in multiple pregnancies (as with the rule-based approach). The coexistence of multiple structures from the same entity (e.g., the fetus) should be classified as a singleton pregnancy.

[0140] If the analysis indicates a possible multiple pregnancy, proceed to step 1135. If the analysis does not indicate a multiple pregnancy, proceed to step 1140.

[0141] In step 1130, method 1100 includes using a rule-based expert system to analyze image frames and bounding boxes. For example, the rule-based expert system may include classical image detection and / or optical character recognition algorithms to identify bounding boxes and / or labels, coupled with one or more rules. For example, these rules may be Boolean expressions indicating the presence, absence, or recurrence of specific features, and / or their meaning.

[0142] For example, if two non-overlapping heart detection boxes exist in the same frame, that frame can be identified as indicating a multiple pregnancy. Another example is the presence of one axial head and another axial abdomen in the same frame. Another example would be the presence of two non-overlapping heads. Another example would be the presence of a head with a pelvic bone / fetal bladder. Other examples include: 1) two heads, hearts, and hip regions (the same anatomical structures) appearing in the same frame; 2) one axial head and one axial abdomen appearing in the same frame; 3) one axial head and one axial chest appearing in the same frame; 4) one axial head and one axial abdomen appearing in the same frame; 5) one axial head and a hip region identified by the pelvic bone appearing in the same frame; or 6) any two identical or different (non-limb) axial anatomical structures appearing in the same frame, regardless of their location / orientation. The examples provided above do not exhaustively cover the entire set of possible rules for distinguishing multiple pregnancies.

[0143] If the analysis of the image frame indicates a multiple pregnancy, proceed to step 1135. If the analysis does not indicate a multiple pregnancy, proceed to step 1140.

[0144] In step 1135, method 1100 includes determining that the image frame contains features indicative of a multiple pregnancy. The process then returns to step 1110 until all image frames have been analyzed, and then proceeds to step 1150.

[0145] In step 1110, method 1100 includes determining that the image frame does not contain features indicating a multiple pregnancy. Execution then returns to step 1110 until all image frames have been analyzed, and then proceeds to step 1150.

[0146] In step 1150, method 1100 includes determining the number or percentage of image frames in the blind scan protocol indicating multiple pregnancies. The process then proceeds to step 1160.

[0147] In step 1110, method 1100 includes determining whether the quantity or percentage in step 1150 exceeds a threshold 1170. If yes, proceed to step 1180. If no, proceed to step 1190.

[0148] In step 1180, method 1100 includes identifying a suspected multiple pregnancy. Method 1100 is now complete.

[0149] In step 1190, method 1100 includes determining that a multiple pregnancy is not suspected. Method 1100 is now complete.

[0150] Figure 12An ultrasound image frame 1200, according to some aspects of this disclosure, includes a single placental bounding box 1210 and two separate, non-overlapping head bounding boxes 1220, indicating that two fetal heads have been detected. The presence of two fetal heads indicates a twin pregnancy, as described above. Figure 11 As described. Seeing two of the unique features in the same image (e.g., a fetus having only one such anatomical structure, such as a fetus having only one head, one heart, one spine, one bladder, one abdomen, one stomach, etc.) (e.g., two heads, two hearts, two spines, two stomachs, two abdomens, etc.) may indicate a multiple pregnancy. However, it is worth noting that other combinations of features may also indicate a multiple pregnancy if a single fetus would not typically be able to show that combination in a single image. For example, the foot being located close to the head may indicate a multiple pregnancy. Another example is the presence of one axial head and another axial abdomen in the same frame. Another example is the presence of one axial head and another axial chest in the same frame. Another example is the simultaneous presence of one axial head and fetal bladder / pelvic bones in the same frame. Other examples include: 1) the buttocks and the thoracic cavity (heart), 2) one axial head and one axial abdomen in one frame, 3) one axial head and one axial chest in one frame, 4) one axial head and one axial abdomen in one frame, 5) one axial head and a buttocks region identified by the pelvic bones in one frame. The above examples are not exhaustive. Depending on the implementation, an image frame (e.g., image frame 1200) can be output to the user, containing multiple bounding boxes that identify multiple detected fetal anatomical sites. The presence of a single placental bounding box 1210 in this image neither indicates nor excludes a multiple pregnancy. In some instances, the placenta is a part of the mother's fetus (e.g., not a part of the fetal body).

[0151] Figure 13 This is a schematic diagram of an example fetal site geometry mapping method 1300, represented in the form of a hybrid block diagram / flowchart, based on various aspects of this disclosure.

[0152] In step 1315, method 1300 includes receiving image frames 1305 from a single sweep (e.g., intra-sweep cinematic loop) and / or image frames 1310 from multiple sweeps (e.g., inter-sweep cinematic loop), and using an object detector (e.g., a trained machine learning network) to detect fetal parts. If image frame 1305 (from only a single sweep) is used, the steps of method 1300 can be repeated for different sweeps. The output of step 1315 includes image frames 1320 with bounding boxes for fetal parts and image frames 1325 without bounding boxes for fetal parts (e.g., image frames where no fetal parts were detected). Execution then proceeds to step 1330.

[0153] In step 1330, method 1300 includes spatial mapping of the fetal portion bounding boxes. The output of step 1330 includes fetal portion clusters 1335 (described below) in the spatial mapping and distances 1340 between fetal portion clusters (described below). Execution then proceeds to step 1345.

[0154] In step 1345, the outputs 1335 and 1340 of the spatial mapping step 1330 are received by artificial intelligence (e.g., a rule-based expert system as described above). For example, the rule might be the existence of two spatially separated head clusters (some near the start of the sweep, and another near the end of the sweep—which is impossible within a sweep in a singleton pregnancy). Another example is the existence of two head clusters that are spatially separated across sweeps (one cluster near the C1 sweep, and another near the C5 sweep—which is impossible between sweeps in a singleton pregnancy). Similar examples could be constructed for the heart, abdomen, fetal bladder, etc., which should not appear in spatially separated areas within the uterus in the case of a singleton pregnancy.

[0155] In some aspects, alternatively or additionally, artificial intelligence may be or include a trained neural network as described above. Based on fetal location clustering 1335 and distance 1340, the artificial intelligence outputs an indication of multiple pregnancy 1350 or a non-multiple pregnancy 1355. Method 1300 is now complete.

[0156] Figure 14 This is a schematic diagram of a scanning process 1400 according to various aspects of this disclosure. The scanning process 1400 includes horizontal scans or sweeps 1410, labeled C1, C2, and C3, and vertical sweeps 1420, labeled R, M, and L. The horizontal sweeps 1410 and vertical sweeps 1420 are performed within a sweep area 1430 on the mother's abdomen 1440, defined by the upper abdomen 1450 and the cervix 1460, thereby forming a sweep grid 1470. The spatial mapping is such that, for example, if the fetal head is detected in the lower portion of the L vertical sweep, it can also be expected to occur in the left portion of the C1 horizontal sweep.

[0157] Figure 15A A graphical representation of a horizontal scan or horizontal sweep 1500 according to various aspects of this disclosure. The horizontal sweep includes multiple ultrasound images 1510, some of which are images 1520 marked with a first color or pattern indicating no detection; and others of which are images 1530 marked with a second color or pattern indicating the detection of fetal parts (such as the head). Two separate clusters of images 1550 containing fetal body parts, separated by sequences of images 1520 where no fetal body parts were detected, may indicate multiple pregnancies.

[0158] Figure 15B This is a graphical representation 1520 of the distance calculation for a horizontal sweep according to various aspects of this disclosure. The graphical representation 1520 includes a Y-axis 1530, indicating the field of view of the ultrasound probe (e.g., measured in degrees, centimeters, or other units), and an X-axis 1540, indicating the sweep length (e.g., measured in frames, centimeters, or other units). A first detection cluster 1550 (e.g., a cluster of frames that detect fetal body parts (such as the head), a second detection cluster 1560 (e.g., a second cluster of frames that detect fetal body parts (such as the head),) and distances 1570 between clusters (e.g., measured in units along the X-axis) are visible in the graphical representation. Also visible in the figure is a color legend 1580 indicating the probability level of the detection results.

[0159] Figure 16A A graphical representation of vertical scans or vertical sweeps 1600 according to various aspects of this disclosure. A vertical sweep includes multiple ultrasound images 1610, some of which are images 1620 marked with a first color or pattern indicating no detection; some of which are images 1630 marked with a second color or pattern indicating the detection of a fetal site, such as the abdomen; and some of which are images 1640 marked with a third color or pattern indicating the detection of two identical fetal sites, such as two abdomens. The detection of two identical body sites in the same image 1640 may indicate a multiple pregnancy.

[0160] Figure 16B A graphical representation 1650 is provided for distance calculations for vertical sweeps according to various aspects of this disclosure. Graphical representation 1650 includes an X-axis 1660 indicating the field of view of the ultrasound probe (e.g., measured in degrees, centimeters, or other units), and a Y-axis 1670 indicating the sweep length (e.g., measured in frames, centimeters, or other units). A first detection cluster 1680 (e.g., a cluster of frames detecting fetal body parts (such as the head), a second detection cluster 1690 (e.g., a second cluster of frames detecting fetal body parts (such as the head),) and an overlapping region 1685 between the clusters (e.g., measured in units along the X-axis) are visible in the graphical representation. The presence of the overlapping region 1685 indicates that the distance between the two clusters 1680 and 1690 is zero. Also visible are color legends 1692 indicating the detection probability level for a single fetal abdomen detection; and color legends 1694 indicating the detection probability for twin fetal abdomen detection.

[0161] Figure 17 The screen display 1700 is an example of an ultrasound blind scan multiple pregnancy detection system according to various aspects of this disclosure. The screen display includes a horizontal scan representation 1710, a vertical scan representation 1720, a combined horizontal and vertical representation 1730, and a combined representation 1740 superimposed on graphic representations of two fetuses 1750. Figure 17 In the example shown, the horizontal sweep representation 1710 includes two detection clusters for the fetal head 1030 and two detection clusters for the fetal heart 1760. Similarly, the vertical sweep representation 1720 includes two detection clusters for the fetal head 1030 and two detection clusters for the fetal heart 1070.

[0162] Combined representation 1730 contains the same detection clusters as vertical sweep representation 1710 and horizontal sweep representation 1720, resulting in two overlapping head detection regions 1760 and two overlapping heart detection regions 1770. Also visible are color legends 1580 and 1780, indicating the detection probabilities of the fetal head and heart, respectively. As can be seen from combined representations 1730 and 1740, overlapping detection regions 1760 and 1770 are indicated with a higher probability than individual detection regions 1030 and 1760. In other words, the head is more likely to be found in overlapping region 1760, and the heart is more likely to be found in overlapping region 1770.

[0163] In the combined representation 1740 of the graphic representation 1750 showing two fetuses, overlapping areas 1760 coincide with the heads of one fetus and overlapping areas 1770 coincide with the hearts of one fetus. Therefore, graphic display 1700 or portions thereof (e.g., combined representation 1730 or combined representation 1740) can be displayed to the user to indicate the presence of two fetuses in the uterus. Depending on the implementation, alternative or additional indications may be provided, including but not limited to text, symbols, colors, sounds, and / or spoken language.

[0164] Figure 18 The spatial mapping screen display 1800 of the example ultrasound blind scan multiple pregnancy detection system according to various aspects of this disclosure is shown. The overlapping regions 1760A and 1760B for head detection and 1770A and 1770B for heart detection are visible. Distances 1810 are also visible, such as the distance 1820 between head cluster 1760A and heart cluster 1770A, the distance 1825 between head cluster 1760A and heart cluster 1770B, the distance 1830 between head cluster 1760B and heart cluster 1770B, the distance 1835 between heart cluster 1770B and head cluster 1760B, the distance 1840 between heart cluster 1770A and heart cluster 1770B, and the distance 1850 between head cluster 1760A and head cluster 1760B.

[0165] Expert systems or trained machine learning models can use the distance between two detected features to determine the likelihood that the two features belong to the same fetus or two different fetuses. For example, two heart clusters that are very close together may represent two detections of the same heart and therefore may belong to the same fetus (or at least it cannot be determined whether two fetuses are present). However, if the distance between two heart clusters exceeds a threshold, they are unlikely to belong to the same fetus. Similarly, the distance between heart and head clusters from a single fetus is expected to be within a specific range. Hearts and heads outside this range (e.g., too close or too far apart) are unlikely to be parts of the same fetus and may therefore indicate that there are two or more fetuses in the uterus.

[0166] Depending on the implementation, the distance between clusters, the distance between clusters and overlapping regions can be calculated, or only the distance between overlapping regions can be calculated.

[0167] Figure 19 This is a schematic diagram of an example fetal site geometric sequence method 1900, represented in the form of a mixed block diagram / flowchart, based on various aspects of this disclosure.

[0168] In step 1930, method 1900 includes receiving image frames 1910 from a single sweep (e.g., intra-sweep cinematic loop) and / or image frames 1920 from multiple sweeps (e.g., inter-sweep cinematic loop), and using an object detector (e.g., a trained machine learning network) to detect fetal parts. If image frame 1910 (from only a single sweep) is used, the steps of method 1900 can be repeated for different sweeps. The output of step 1930 includes image frame 1940 with fetal part bounding boxes and labels and image frame 1950 without fetal part bounding boxes and labels (e.g., image frames where no fetal parts were detected). Execution then proceeds to step 1960.

[0169] In step 1960, method 1900 includes label-based text recognition, assembling a text sequence of anatomical labels from image frame 1940 in the order in which the labels appear during the sweep. For example, for frames where no anatomical structures were detected, the text sequence may be empty; and for frames where anatomical structures were detected, the text sequence may contain one or more labels. The individual frames in the text sequence can be separated by delimiters (e.g., commas, semicolons, colons, etc.). For example, the text sequence may take the form: “[Frame 1 Anatomical Label]; [Frame 2 Anatomical Label]; ……”. One example is “Heart, Head; Heart, Head; ... Other delimiters can be used to join multiple sweep sequences together to create a sequence of the entire inspection. Another example of merging multiple sweep sequences is to create a new sequence where the first entry is the first entry of C1, the second entry is the first entry of C2, and so on. However, other methods of combining sequences across sweeps are possible and fall within the scope of this disclosure.

[0170] The text sequence is then received by a sequence model 1970, which may be, for example, an expert system or a trained machine learning model (e.g., a hidden Markov model, a recurrent neural network, etc.). The model detects (a) whether the same anatomical structure appears multiple times in a single frame, and / or (b) whether the text sequence represents an unlikely arrangement of anatomical features of a single fetus (e.g., the heart is detected twice, with several empty frames in between), and / or (c) whether multiple scans of the text sequence collectively represent an unlikely arrangement of anatomical features of a single fetus (e.g., the heart is detected in both the R and L scans, but not in the M scan). One example of how sequence models can help detect multiple pregnancies is assigning a low score to a sequence such as “head; head ...

[0171] Figure 20 This is a schematic diagram illustrating an example fetal site geometric sequence method 2000, represented in block diagram form, based on various aspects of this disclosure. Figure 20 In the example shown, each sweep 2005, 2015, 2025, 2035, 2045, and 2055 contains bounding boxes and their associated labels added by the object detector. The labels are then stripped and delimited frame by frame (e.g., separated by a delimiter such as a semicolon after each frame), with undetected frames marked with empty frames, spaces, the word "empty," or other indicators, to assemble the corresponding anatomical label text sequence 2010, 2020, 2030, 2040, 2050, and 2060 representing the sweep.

[0172] Figure 21A This is a schematic diagram illustrating an example fetal site geometric sequence method 2100, represented in block diagram form, based on various aspects of this disclosure. Figure 21A In the example shown, each horizontal sweep 2005, 2015, 2025 contains bounding boxes and their associated labels added by the object detector. Then, following the method described above, the labels are stripped and separated frame by frame to assemble individual text sequences 2110 of the corresponding dissected labels, which encode the object detection results of all three horizontal sweeps in the order they were captured.

[0173] Figure 21B This is a schematic diagram illustrating an example fetal site geometric sequence method 2120, represented in block diagram form, based on various aspects of this disclosure. Figure 21B In the example shown, each vertical sweep 2035, 2045, 2055 contains bounding boxes and their associated labels added by the object detector. Then, following the method described above, the labels are stripped and separated frame by frame to assemble individual text sequences 2110 of the corresponding dissected labels, which, for example, encode the object detection results of all three vertical sweeps in the order they were captured.

[0174] Figure 22 This is a schematic diagram illustrating an example fetal site geometric sequence method 2120, represented in block diagram form, based on various aspects of this disclosure. Figure 22 In the example shown, each sweep 2005, 2015, 2025, 2035, 2045, and 2055 contains bounding boxes and their associated labels added by the object detector. Then, following the method described above, the labels are stripped and separated frame by frame to assemble individual text sequences of the corresponding anatomical labels 2110, which, for example, encode the object detection results of all three horizontal sweeps and all three vertical sweeps in the order they were captured.

[0175] and Figure 20 , 21A Other orders of the anatomical labels in the associated text sequences 21B and / or 22 may be used alternatively or supplementarily.

[0176] In some instances, when multiple sweeps are part of a given text sequence (e.g.) Figure 21A , 21B As shown in Figure 22), the order of anatomical tags from multiple scans can be ordered differently than the actual scans performed by the ultrasound probe. For example, in a blind scan protocol, scan C1 may be performed before scan C2 (e.g., Figure 4 The anatomical label portion of the C1 sweep can be placed before the anatomical label portion of the C2 sweep in the text sequence (consistent with the order in which sweeps are physically performed), or it can be placed after the anatomical label portion of the C2 sweep in the text sequence (different from the order in which sweeps are physically performed).

[0177] For example, the order of anatomical labels in a text sequence may or may not correspond to the movement order / direction of the ultrasound probe in a given blind ultrasound scan (e.g., ...). Figure 4(As shown). For example, a C1 scan might involve the ultrasound probe starting on the patient's right side and moving towards the patient's left side (or vice versa in other cases). The order of the anatomical labels can match the direction of probe movement (labels from the patient's right side appear relatively earlier in the text sequence, and labels from the patient's left side appear relatively later in the text sequence), or it can be the opposite (labels from the patient's left side appear relatively earlier in the text sequence, and labels from the patient's right side appear relatively later in the text sequence).

[0178] When multiple sweeps are part of a given text sequence (e.g.) Figure 21A , 21B (As shown in Figure 22), multiple sweeping anatomical tags can be used in the same or different orders. For example, in Figure 21A In the process, the anatomical labels for each of the C1, C2, and C3 scans can be matched with the direction of probe movement. Then, the text sequence takes the form of "[C1 anatomical label – patient right to patient left], [C2 anatomical label – patient right to patient left], [C3 anatomical label – patient right to patient left]". Similarly, this can also be applied to... Figure 21B The associated text sequences are processed (e.g., "[R Anatomical Label – Patient Bottom to Patient Top], [M Anatomical Label – Patient Bottom to Patient Top], [L Anatomical Label – Patient Bottom to Patient Top]"). For those related to... Figure 22 Similar operations can be performed on associated text sequences ("[R anatomical label – patient bottom to patient top], [M anatomical label – patient bottom to patient top], [L anatomical label – patient bottom to patient top], [C1 anatomical label – patient right side to patient left side], [C2 anatomical label – patient right side to patient left side], [C3 anatomical label – patient right side to patient left side]").

[0179] In other cases, for example, the order of the anatomical labels in the C2 sweep can be interchanged relative to the order of the anatomical labels in the C1 and C3 sweeps. The text sequence then takes the form of “[C1 anatomical labels – patient right to patient left], [C2 anatomical labels – patient left to patient right], [C3 anatomical labels – patient right to patient left]”. This can be done to attempt to associate the physical locations of the anatomical labels at the beginning and / or end of the C1, C2, and / or C3 sections of the text sequence. For example, when the order of the anatomical labels in the C2 sweep is interchanged, the physical location of the anatomical label at the end of the C1 sweep text sequence (on the patient's left side) is closer to and / or otherwise closer to the location of the anatomical label at the beginning of the C2 sweep text sequence (on the same side of the patient's left side, higher up in the patient's abdomen). In contrast, when the order of the anatomical tags in the C2 sweep is consistent with the order of the anatomical tags in the C1 and C3 sweeps, the physical location of the anatomical tag from the end of the C1 sweep portion of the text sequence (on the patient's left side) is farther from the location of the anatomical tag from the beginning of the C2 sweep portion of the text sequence (both on different sides of the patient's right side, and higher in the patient's abdomen). Similarly, when the order of the anatomical tags in the C2 sweep is reversed, the physical location of the anatomical tag at the end of the C2 sweep text sequence (on the patient's right side) is closer to and / or otherwise closer to the location of the anatomical tag at the beginning of the C3 sweep text sequence (on the same side of the patient's right side, higher in the patient's abdomen). Figure 21B The associated text sequence can also take the form of "[R anatomical label – patient bottom to patient top], [M anatomical label – patient top to patient bottom], [L anatomical label – patient bottom to patient top]". For [the context of the previous sentence, the last part is missing - likely a formatting error]. Figure 22 Related text sequences can be treated similarly. For example, Figure 22 The order of the anatomical labels can follow a winding path, making the text sequence resemble "[R anatomical label – patient bottom to patient top], [M anatomical label – patient top to patient bottom], [L anatomical label – patient bottom to patient top], [C3 anatomical label – patient left side to patient right side], [C2 anatomical label – patient right side to patient left side], [C3 anatomical label – patient left side to patient right side]".

[0180] Those skilled in the art who are familiar with the content of this article will readily understand that a blind ultrasound scan system for detecting multiple pregnancies allows untrained or minimally trained users to perform blind ultrasound scan protocols, thereby obtaining high-quality anatomical images, including the automated detection of potential health conditions such as multiple pregnancies (MG). This can improve accuracy, increase clinician confidence in the results, and potentially improve health outcomes and / or reduce overall healthcare costs. Potential benefits include the detection of multiple pregnancies through blind scans performed by novice ultrasound users. For centers with high ultrasound turnover, the solution can be a rapid initial examination scan for patient triage to a more detailed obstetric scan; and can provide or support referral of individuals diagnosed with multiple pregnancies to a tertiary medical center for further diagnosis and management. Early detection of multiple pregnancies can be extremely helpful for subsequent follow-up and monitoring of the pregnancy.

[0181] The systems, methods, and devices described herein can be applied to point-of-care and handheld ultrasound applications, such as the Philips Lumify system. The ultrasound blind-scan multiple pregnancy detection system can be used in any handheld imaging application, including but not limited to obstetric and cardiac ultrasound examinations. The ultrasound blind-scan multiple pregnancy detection system can be deployed on handheld mobile ultrasound devices as well as portable or cart-mounted ultrasound systems. The ultrasound blind-scan multiple pregnancy detection system can be used in a variety of locations, including emergency rooms, ambulances, accident scenes, and homes. These applications can also be extended to other fields.

[0182] This system can perform detections through its functions and outputs, such as reporting detected health conditions, like multiple pregnancies. This invention enhances the value of ultrasound in obstetric applications, especially for users with minimal training.

[0183] Therefore, the logical operations constituting the various aspects of the technology described herein can be referred to as operations, steps, objects, layers, elements, components, algorithms, or modules, respectively. Furthermore, it should be understood that unless expressly specified in the claims or the language of the claims inherently requires a particular order, these can occur, be performed, or arranged in any order.

[0184] All directional references, such as up, down, inside, outside, upward, downward, left, right, lateral, front, back, top, bottom, above, below, vertical, horizontal, clockwise, counterclockwise, proximal, and distal, are for identification purposes only to aid the reader in understanding the claimed object and do not impose limitations, particularly regarding the location, orientation, or purpose of the said ultrasound blind scan multiple pregnancy detection system. Connection references, such as attached, coupled, connected, combined, or “connected to,” should be interpreted broadly and may include intermediate elements between sets of elements and relative movement between elements, unless otherwise indicated. Therefore, a connection reference does not necessarily mean that two elements are directly connected and have a fixed relationship with each other. The term “or” should be interpreted as “and / or,” not “exclusive or.” The word “comprising” does not exclude other elements or steps, and the words “a” or “an” do not exclude multiple. Unless otherwise stated in the claims, the numerical values ​​are illustrative only and should not be considered restrictive.

[0185] The foregoing description, examples, and data provide a complete description of the structure and use of exemplary aspects of the ultrasound blind scan multiple pregnancy detection system as defined in the claims. While various aspects of the claimed object have been described in a particular degree of detail above, or with reference to one or more aspects, those skilled in the art can make many modifications to the disclosed aspects without departing from the spirit or scope of the claimed object.

[0186] Other aspects have also been conceived. All content contained in the above description and drawings should be construed as illustrative of specific aspects only and is not intended to be limiting. Changes in detail or structure may be made without departing from the essential elements of the subject matter as defined in the following claims.

Claims

1. A system comprising: A processor configured to communicate with an ultrasound probe, wherein the processor is configured to: The ultrasound probe is controlled to acquire multiple ultrasound image frames during a blind scan protocol for pregnant patients. The plurality of ultrasound image frames are provided as input to a deep learning network trained to detect at least one of maternal or fetal anatomical structures. Generate a detection for at least one of the following as the output of the deep learning network: The intertwin septum within the multiple ultrasound image frames; or Multiple fetal anatomical sites within the multiple ultrasound image frames; The detection of at least one of the intertwin septum or the plurality of fetal anatomical sites is used to determine whether the pregnancy includes a multiple pregnancy; and The processor provides a display that communicates with it with an output indicating whether the pregnancy includes the multiple pregnancy.

2. The system of claim 1, wherein, The processor is configured to perform the determination of whether the pregnancy includes the multiple pregnancy based on at least one of the following: The detection of the intertwin separator membrane; The multiple fetal anatomical sites are simultaneously present within a single ultrasound image frame; Geometric mapping of the multiple fetal anatomical sites; or The geometric sequence of the multiple fetal anatomical sites.

3. The system of claim 2, wherein, In order to perform the determination of whether the pregnancy includes the multiple pregnancy, the processor is configured to combine the results of at least two of the following: The detection of the intertwin separator membrane; The multiple fetal anatomical sites are simultaneously present within the single ultrasound image frame; The geometric mapping of the plurality of fetal anatomical sites; or The geometric sequence of the plurality of fetal anatomical sites.

4. The system according to claim 2, in, The output of the deep learning network includes multiple detections of the intertwin septum in the plurality of ultrasound image frames. In order to perform the determination of whether the pregnancy includes a multiple pregnancy based on the detection of the intertwin septum in the plurality of ultrasound image frames, the processor is configured to: Determine the number of the plurality of ultrasound image frames detected with the intertwin septum membrane; Compare the quantity with the threshold quantity; and When the number exceeds the threshold number, the pregnancy is determined to include the multiple pregnancy.

5. The system according to claim 4, wherein, The output provided to the display includes: Having at least one ultrasound image frame of the detection of the intertwined septum; and A bounding box superimposed on the at least one ultrasound image frame to identify the intertwin septum.

6. The system according to claim 2, in, The output of the deep learning network for a single ultrasound image frame includes multiple detections of the plurality of fetal anatomical sites. In order to perform the determination of whether the pregnancy includes a multiple pregnancy based on the simultaneous presence of the multiple fetal anatomical sites within the single ultrasound image frame, the processor is configured to: The multiple detections are provided as input to at least one of a statistical model or a rule-based expert system; and The determination of a multiple pregnancy indicated by the single ultrasound image frame is generated as the output of at least one of the statistical model or the rule-based expert system.

7. The system according to claim 6, wherein, In order to perform the determination of whether the pregnancy includes a multiple pregnancy based on the simultaneous presence of the multiple fetal anatomical sites within the single ultrasound image frame, the processor is configured to: The determination that a single ultrasound image frame indicates a multiple pregnancy is repeated for each of the plurality of ultrasound image frames; Determine the number of the plurality of ultrasound image frames indicating the multiple pregnancy; Compare the quantity with the threshold quantity; and When the number exceeds the threshold number, the pregnancy is determined to include the multiple pregnancy.

8. The system according to claim 7, wherein, The output provided to the display includes: The single ultrasound image frame; and Multiple bounding boxes superimposed on the single ultrasound image frame to identify the multiple fetal anatomical sites.

9. The system according to claim 2, in, The output of the deep learning network includes multiple detections of the multiple fetal anatomical sites in the multiple ultrasound image frames. In order to perform the determination of whether the pregnancy includes a multiple pregnancy based on the geometric mapping of the plurality of fetal anatomical sites, the processor is configured to: Generate spatial mappings of the multiple fetal anatomical sites; and The determination of the pregnancy, including the multiple pregnancy, is performed using a rule-based expert system and the spatial mapping.

10. The system according to claim 9, wherein, In order to perform the determination of whether the pregnancy includes a multiple pregnancy based on the geometric mapping of the plurality of fetal anatomical sites, the processor is configured to: Detecting multiple regions having the multiple fetal anatomical sites in the spatial mapping; Determine at least one distance between the plurality of regions; The plurality of regions and the at least one distance are provided as inputs to the rule-based expert system; and The determination of the pregnancy, including the multiple pregnancy, is generated as the output of the rule-based expert system.

11. The system according to claim 9, wherein, The output provided to the display includes: The spatial mapping.

12. The system according to claim 2, in, The output of the deep learning network includes multiple detections of the multiple fetal anatomical sites in the multiple ultrasound image frames. In order to perform the determination of whether the pregnancy includes a multiple pregnancy based on the geometric sequence of the plurality of fetal anatomical sites, the processor is configured to: Generate a text sequence representing anatomical labels for the plurality of fetal anatomical sites and the plurality of ultrasound image frames; The text sequence is provided as input to the sequence model; and The determination of the pregnancy, including the multiple pregnancy, is generated as the output of the sequence model.

13. The system according to claim 12, in, The plurality of ultrasound image frames include the first and second scans of the blind scan protocol. The physical location associated with the anatomical tag at the end of the first portion of the first sweep in the text sequence is close to the physical location associated with the anatomical tag at the beginning of the second portion of the second sweep in the text sequence.

14. The system according to claim 1, further comprising the ultrasonic probe.

15. A method comprising: The processor controls an ultrasound probe that communicates with the processor to acquire multiple ultrasound image frames during a blind scan protocol on a pregnant patient. The processor provides the plurality of ultrasound image frames as input to a deep learning network, which is trained to detect the patient's anatomical structures. The processor generates a detection of at least one of the following as the output of the deep learning network: The intertwin septum within the multiple ultrasound image frames; or Multiple fetal anatomical sites within the multiple ultrasound image frames; The processor determines whether the pregnancy includes a multiple pregnancy by detecting at least one of the intertwin septum or the plurality of fetal anatomical sites. as well as The processor provides a display communicating with the processor with a determined output indicating whether the pregnancy includes a multiple pregnancy.

16. The method according to claim 15, wherein, Determining whether the pregnancy includes a multiple pregnancy is based on at least one of the following: The detection of the intertwin separator membrane; The multiple fetal anatomical sites are simultaneously present within a single ultrasound image frame; Geometric mapping of the multiple fetal anatomical sites; or The geometric sequence of the multiple fetal anatomical sites.

17. The method according to claim 16, in, The output of the deep learning network includes multiple detections of the intertwin septum in the plurality of ultrasound image frames. Determining whether a pregnancy includes a multiple pregnancy based on the detection of the intertwin septum in the plurality of ultrasound image frames includes: Determine the number of the plurality of ultrasound image frames detected with the intertwin septum membrane; Compare the quantity with the threshold quantity; and When the number exceeds the threshold number, the pregnancy is determined to include the multiple pregnancy.

18. The method according to claim 16, in, The output of the deep learning network for a single ultrasound image frame includes multiple detections of the plurality of fetal anatomical sites. Determining whether a pregnancy includes a multiple pregnancy based on the simultaneous presence of multiple fetal anatomical sites within a single ultrasound image frame includes: The multiple detections are provided as input to at least one of a statistical model or a rule-based expert system; and The determination of a multiple pregnancy indicated by the single ultrasound image frame is generated as the output of at least one of the statistical model or the rule-based expert system.

19. The method according to claim 16, in, The output of the deep learning network includes multiple detections of the multiple fetal anatomical sites in the multiple ultrasound image frames. Determining whether a pregnancy includes a multiple pregnancy based on the geometric mapping of the multiple fetal anatomical sites includes: Generate spatial mappings of the multiple fetal anatomical sites; and The determination of the pregnancy, including the multiple pregnancy, is performed using a rule-based expert system and the spatial mapping.

20. The method according to claim 16, in, The output of the deep learning network includes multiple detections of the multiple fetal anatomical sites in the multiple ultrasound image frames. Determining whether the pregnancy includes a multiple pregnancy based on the geometric sequence of the multiple fetal anatomical sites includes: Generate a text sequence representing anatomical labels for the plurality of fetal anatomical sites and the plurality of ultrasound image frames; The text sequence is provided as input to the sequence model; and The determination of the pregnancy, including the multiple pregnancy, is generated as the output of the sequence model.

Images