System for generating three-dimensional (3D) rendered model of a uterus along with uterine structural abnormalities
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Filing Date
- 2026-01-28
- Publication Date
- 2026-08-06
Smart Images
Figure IN2026050149_06082026_PF_FP_ABST
Abstract
Description
SYSTEM FOR GENERATING THREE-DIMENSIONAL (3D) RENDERED MODEL OF A UTERUS ALONG WITH UTERINE STRUCTURAL ABNORMALITIES BACKGROUNDTechnical Field
[0001] The embodiments herein generally relate to medical imaging and more particularly to the system and method for generating three-dimensional (3D) rendered model of a uterus along with uterine structural abnormalities.Description of the Related Art
[0002] Uterine fibroids are non-cancerous growths that develop in or around the uterus and commonly affect women of reproductive age. Adenomyomas are benign tumours composed of endometrial cells that are commonly found in the myometrium while Polyps are soft non-cancerous growths found in the endometrial cavity of the uterus. These growths can result in symptoms such as heavy menstrual bleeding, pelvic pain, and fertility issues. Accurate detection, characterization, and spatial localization of uterine pathologies relative to uterine anatomy and the endometrium are critical for effective clinical management and treatment planning.
[0003] Existing methods for managing uterine pathologies include non-invasive imaging techniques, minimally invasive treatments, surgical interventions, and medical therapy. The non-invasive imaging techniques, such as ultrasound and Magnetic Resonance Imaging (MRI), are commonly used to detect and evaluate the fibroids. The minimally invasive treatments include Uterine Artery Embolization (UAE) Focused Ultrasound Surgery (FUS) and Ablation. The surgical interventions, such as myomectomy and hysterectomy, are selected depending on the patient’s condition. Medical therapy is often used to manage symptoms. The choice of treatment depends on fibroid size, location, symptom severity, and the patient’s fertility goals. Accurate spatial understanding of uterine anatomy and pathology distribution is essential for selecting appropriate therapies, guiding interventions, and counselling patients regarding procedural risks and outcomes.
[0004] Traditional ultrasound imaging, although widely accessible and non-invasive, has significant limitations. Small or deeply located fibroids may be missed, and results are highly dependent on the skill of an operator, which can lead to variability. Ultrasound provides limited information about fibroid composition and morphology, and it often lacks detailed spatialrepresentation of the fibroids relative to the uterine structure. MRI provides higher-resolution imaging than the ultrasound and can more accurately characterize fibroid size, location, and tissue properties. Despite the advantages, MRI is expensive, less widely available, and time-consuming. Some patients, such as those with metal implants or claustrophobia, cannot undergo MRI, which restricts its practical use in routine diagnosis and monitoring.
[0005] Existing fibroid and uterine pathology mapping methods aim to identify and locate the pathologies within the uterus. The methods often face accuracy challenges due to difficulties in understanding the spatial relationships between the fibroids and a uterine wall, often lacking a unified spatial view. The fibroids may vary in position, shape, and size, and in some cases, are difficult to distinguish from surrounding tissue. The limited field of view can result in missing lateral fibroids. These can lead to errors in determining fibroid number, size, or location, which can adversely impact treatment planning and outcomes.
[0006] Despite the availability of the imaging and mapping techniques, challenges remain in providing accurate, detailed, and clinically useful representations of the uterine fibroids and other uterine pathologies. Limitations in imaging resolution, operator dependency, accessibility, real-time feedback, and spatial accuracy underscore the ongoing need for more reliable and precise diagnostic and mapping methods.
[0007] Accordingly, there remains a need for improved methods and systems that provide more accurate, detailed, and clinically useful representations of the uterus along with uterine structural pathologies, while overcoming the limitations of existing imaging and mapping approaches.SUMMARY
[0008] In view of the foregoing, embodiments herein provide a method for generating a three-dimensional (3D) rendered model of a uterus with an endometrium along with other uterine pathologies. The method includes (1) receiving multimodal medical images of the uterus, the endometrium, and other uterine pathologies, from a plurality of user devices, where the multimodal medical images includes one of two-dimensional (2D) ultrasound volume data or 2D magnetic resonance imaging (MRI) volume data; (2) extracting a plurality of frames from each imaging plane of the multimodal medical images, where the imaging planes includes two anatomical planes; (3) determining an anatomical orientation and spatial position of the uterine pathologies with respect to the uterus and the endometrium within the plurality of frames; (4) selecting one ormore optimal frames from the plurality of frames associated with each of the two anatomical planes for the uterus, the endometrium, and the other uterine pathologies, where each of the one or more selected optimal frames of respective anatomical structure; (5) segmenting uterine boundaries, endometrial boundaries, and other uterine pathology boundaries within the one or more selected optimal frames; (6) mapping the segmented uterine boundaries, the endometrial boundaries, and the other uterine pathology boundaries, from the one or more selected optimal frames into three-dimensional spatial coordinate system based on the depth-scale normalization and alignment of the imaging planes; and (7) generating a 3D rendered model of the uterus, the endometrium, and the other uterine pathologies with accurate dimensional and spatial information for the uterine pathologies by reconstructing a volumetric representation from mapped volumetric spatial coordinates.
[0009] A system and method for generating an anatomically accurate three-dimensional rendered model of a uterus along with uterine structural abnormalities using two-dimensional ultrasound and magnetic resonance imaging data. By extracting image frames from multiple anatomical planes, normalizing embedded depth-scale information, and mapping segmented uterine and abnormality boundaries into a unified three-dimensional spatial coordinate system, the system enables reliable volumetric reconstruction without requiring dedicated three-dimensional imaging acquisition systems.
[0010] The system improves consistency and reduces operator dependency through the use of machine-learning-based frame selection, localization, and boundary segmentation. Automated identification of diagnostically relevant frames and region-of-interest-based segmentation enables accurate delineation of uterine structures, endometrial layers, and uterine structural abnormalities across variations in image quality, anatomical presentation, and acquisition conditions.
[0011] The three-dimensional rendered model generated by the system provides enhanced visualization and precise spatial and dimensional assessment of uterine anatomy and uterine structural abnormalities compared to conventional two-dimensional image interpretation. The system supports accurate evaluation of endometrial thickness, size and location of uterine structural abnormalities, and generation of virtual hysteroscopy visualizations, thereby improving diagnostic confidence, pre-procedural planning, and overall clinical workflow efficiency.
[0012] In some embodiments, the method including the detection and segmentation of the uterine structures, endometrium structures and uterine pathologies use a machine learning pipelineincluding (i) one or more optimal frames is selected using a detection machine-learning model that is trained to classify the plurality of frames into frames including uterine pathology information and to identify one or more frames corresponding to optimal cross-sectional representation of the other uterine pathologies, (ii) a localization machine-learning model configured to automatically generate bounding-box prompts corresponding to regions of interest associated with uterine structures, endometrium structures and the other uterine pathologies within the one or more selected optimal frames; and (iii) a boundary-segmentation machine-learning model trained using a prompt-based segmentation architecture and a training dataset including medical images annotated with uterine boundaries, endometrial boundaries, and the other uterine pathology boundaries, the boundary-segmentation machine-learning model being configured to segment the uterine boundaries, the endometrial boundaries, and the other uterine pathology boundaries within the regions of interest defined by the bounding-box prompts.
[0013] In some embodiments, the method including the detection machine-learning model includes a deep-learning-based classification model implemented using a residual neural network (ResNet) architecture, and where the localization machine-learning model includes an objectdetection model implemented using a You-Only-Look-Once (YOLO) architecture.
[0014] In some embodiments, the method including the boundary-segmentation machinelearning model includes a Medical Segment Anything Model (MedSAM), configured to receive the bounding-box prompt corresponding to a detected uterine pathology and to generate segmentation masks for the uterine boundaries, the endometrial boundaries, and the other uterine pathology boundaries within the bounding-box prompt.
[0015] In some embodiments, the method includes automatically extracting a depth scale embedded in each frame and converting into real anatomical distances for accurate volumetric mapping.
[0016] In some embodiments, the method includes generating the three-dimensional rendered model includes combining segmented boundaries from the anatomical planes to reconstruct volumetric geometry of the uterus, the endometrium, and the other uterine pathologies.
[0017] In some embodiments, the method includes the three-dimensional rendered anatomical model of the uterus including both a uterine structure and an endometrial layer, thereby enabling visualization and assessment of endometrial thickness, size, and accurate spatial characteristics of uterine pathologies.
[0018] In some embodiments, the segmented endometrial boundaries are employed to generate a virtual hysteroscopy visualization including a navigable fly-through of a reconstructed uterine cavity.
[0019] In an aspect, embodiments herein provide a system for generating a three-dimensional (3D) rendered model of a uterus with an endometrium along with other uterine pathologies. The system includes a memory and a processor communicatively connected to the memory and configured to (1) receive multimodal medical images of the uterus, the endometrium, and other uterine pathologies, from a plurality of user devices, where the multimodal medical images includes one of two-dimensional (2D) ultrasound volume data or 2D magnetic resonance imaging (MRI) volume data; (2) extract a plurality of frames from each imaging plane of the multimodal medical images, where the imaging planes includes two anatomical planes; (3) determine an anatomical orientation and spatial position of the uterine pathologies with respect to the uterus and the endometrium within the plurality of frames; (4) select one or more optimal frames from the plurality of frames associated with each of the two anatomical planes for the uterus, the endometrium, and the other uterine pathologies, where each of the one or more selected optimal frames of respective anatomical structure; (5) segment uterine boundaries, endometrial boundaries, and other uterine pathology boundaries within the one or more selected optimal frames; (6) map the segmented uterine boundaries, the endometrial boundaries, and the other uterine pathology boundaries, from the one or more selected optimal frames into three-dimensional spatial coordinate system based on the depth-scale normalization and alignment of the imaging planes; and (7) generate a 3D rendered model of the uterus, the endometrium, and the other uterine pathologies with accurate dimensional and spatial information for the uterine pathologies by reconstructing a volumetric representation from mapped volumetric spatial coordinates.
[0020] In some embodiments, the system including the detection and segmentation of the uterine structures, endometrium structures and uterine pathologies use a machine learning pipeline including (i) one or more optimal frames is selected using a detection machine-learning model that is trained to classify the plurality of frames into frames including uterine pathology information and to identify one or more frames corresponding to optimal cross-sectional representation of the other uterine pathologies, (ii) a localization machine-learning model configured to automatically generate bounding-box prompts corresponding to regions of interest associated with uterine structures, endometrium structures and the other uterine pathologies within the one or moreselected optimal frames; and (iii) a boundary-segmentation machine-learning model trained using a prompt-based segmentation architecture and a training dataset including medical images annotated with uterine boundaries, endometrial boundaries, and the other uterine pathology boundaries, the boundary-segmentation machine-learning model being configured to segment the uterine boundaries, the endometrial boundaries, and the other uterine pathology boundaries within the regions of interest defined by the bounding-box prompts.
[0021] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications.BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The embodiments herein will be better understood from the following detailed description with reference to the drawings, in which:
[0023] FIG. 1 illustrates a block diagram of system 100 for generating a three-dimensional (3D) rendered model of a uterus along with uterine structural abnormalities according to some embodiment herein;
[0024] FIG. 2 illustrates a block diagram of the 3D uterus model generating server according to some embodiment herein;
[0025] FIG. 3 illustrates a block diagram of a MINV device 300 utilizing 2D ultrasound volume data or 2D MRI volume data according to some embodiments herein;
[0026] FIG. 4 illustrates a 3D-rendered model 400 of a uterus with mapping uterine pathologies according to some embodiments;
[0027] FIG. 5 illustrates a fly through feature that enables the user to view an inside view 500 of the uterine cavity which enables a virtual hysteroscopy according to some embodiments;
[0028] FIG. 6 are flow diagrams that illustrates a method for generating the three-dimensional (3D) rendered model of the uterus along with uterine structural abnormalities according to some embodiment herein; and
[0029] FIG. 7 is representative hardware environment for practicing the embodimentsherein, with reference to FIGS. 1 through 6 in accordance with the embodiments herein.DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
[0030] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.
[0031] As mentioned, there remains a need for a system and a method for generating three-dimensional (3D) rendered model of a uterus along with uterine structural abnormalities. Referring now to the drawings, and more particularly, FIGS. 1 through 7, where similar reference characters denote corresponding features consistently throughout the figures, preferred embodiments are shown.
[0032] The term “Multimodal medical images” refers to medical imaging data acquired from two or more imaging modalities, including two-dimensional ultrasound volume data and two-dimensional magnetic resonance imaging volume data, representing anatomical structures of the uterus, the endometrium, and other uterine pathologies.
[0033] The term “Anatomical plane” refers to a two-dimensional imaging plane representing a cross-sectional view of anatomical structures within medical imaging data, used for extracting frames and reconstructing three-dimensional anatomical geometry.
[0034] The term “Frame” refers to a discrete two-dimensional image slice extracted from an imaging plane of multimodal medical images, preserving anatomical structural information for subsequent detection, segmentation, and reconstruction.
[0035] The term “Optimal frame” refers to a selected frame from an anatomical plane that provides an optimal cross-sectional representation of an anatomical structure or uterine pathology suitable for accurate boundary segmentation and volumetric reconstruction.
[0036] The term “Detection machine-learning model” refers to a trained classification model configured to analyze extracted frames and classify the frames into frames comprising uterine pathology information and frames without pathological information.
[0037] The term “Localization machine-learning model” refers to a trained objectdetection model configured to automatically generate bounding-box prompts corresponding to regions of interest associated with uterine structures, endometrial structures, and uterine pathologies within selected frames.
[0038] The term “Boundary-segmentation machine-learning model” refers to a trained segmentation model configured to generate segmentation masks corresponding to uterine boundaries, endometrial boundaries, and uterine pathology boundaries within regions of interest defined by bounding-box prompts.
[0039] The term “Volumetric spatial coordinates” refers to three-dimensional coordinate values representing spatial positions of segmented anatomical boundaries within a reconstructed volumetric representation of the uterus, the endometrium, and other uterine pathologies.
[0040] The term “Medical Segment Anything Model (MED-SAM)” refers to a specialized segmentation model adapted for medical imaging, capable of generating segmentation masks from automatically generated prompts, enabling efficient identification of uterine, endometrial, and lesion boundaries.
[0041] The term “Bounding-box prompt” refers to an automatically generated rectangular region that surrounds a detected anatomical structure. This bounding box serves as the input prompt for the segmentation model to refine and predict accurate anatomical boundaries.
[0042] The term “Three-dimensional rendered model” refers to a reconstructed three-dimensional anatomical representation generated from mapped volumetric spatial coordinates, comprising spatially accurate geometry of the uterus, the endometrium, and other uterine pathologies.
[0043] The term “Anatomical orientation and spatial position” refers to a determined alignment and relative spatial relationship of uterine pathologies with respect to the uterus and the endometrium within extracted imaging frames and across anatomical planes.
[0044] The term “Residual neural network (ResNet)” refers to a deep-learning-based classification architecture comprising residual learning blocks, configured to perform frame-level classification for detection of uterine pathology information within extracted imaging frames.
[0045] The term “You-Only-Look-Once (YOLO)” refers to an object-detection architecture configured to perform single -pass detection and localization of anatomical structures by predicting bounding boxes and probability scores within an imaging frame.
[0046] The term “Segmentation mask” refers to a pixel-wise labeled representation generated by a segmentation model, identifying boundary regions corresponding to uterine boundaries, endometrial boundaries, and uterine pathology boundaries within an imaging frame.
[0047] The term “Volumetric reconstruction” refers to the process of reconstructing a three-dimensional anatomical geometry by combining segmented boundaries from at least two anatomical planes and mapping the boundaries into a three-dimensional spatial coordinate system.
[0048] The term “Virtual hysteroscopy” refers to a computer-generated navigable visualization of a reconstructed uterine cavity generated from segmented endometrial boundaries, enabling internal cavity visualization without invasive instrumentation.
[0049] FIG. 1 illustrates a block diagram of system 100 for generating a three-dimensional (3D) rendered model of a uterus along with uterine structural abnormalities according to some embodiment herein. The system 100 includes a machine image navigating and visualization (MINV) device 106, a network 108, and a multimodal image processing and 3D modelling server 110. The multimodal image processing and 3D modelling server 110 includes a memory and a processor configured to generate a three-dimensional rendered model of the uterus, the endometrium, and other uterine pathologies from multimodal medical images.
[0050] The MINV device 106 receives multimodal medical images of the uterus, the endometrium, and other uterine pathologies from a plurality of user devices. In some embodiments, the multimodal medical images include the two-dimensional ultrasound volume data 102 or the two-dimensional magnetic resonance imaging volume data 104. The MINV device 106 transmits the received multimodal medical images to the multimodal image processing and 3D modelling server 110 through the network 108 for automated processing and three-dimensional reconstruction.
[0051] The multimodal medical images include multiple anatomical imaging planes. Upon receiving the multimodal medical images, the multimodal image processing and 3D modelling server 110 extracts a plurality of frames from each imaging plane, where the imaging planes include at least two anatomical planes. Each extracted frame represents a discrete two-dimensional anatomical slice preserving uterine structural information, endometrial information, and uterine pathology information.
[0052] The network 108 facilitates data communication between the MINV device 106 and the multimodal image processing and 3D modelling server 110. In some embodiments, thenetwork 108 includes a wired network. In some embodiments, the network 108 is a wireless network. In some embodiments, the network 108 is a combination of a wired network and a wireless network. In some embodiments, network 108 is the Internet.
[0053] The multimodal image processing and 3D modelling server 110 determines an anatomical orientation and spatial position of the uterine pathologies with respect to the uterus and the endometrium within the plurality of extracted frames. In some embodiments, determining anatomical orientation includes classifying each imaging plane and determining spatial relationships among anatomical structures across the imaging planes.
[0054] The multimodal image processing and 3D modelling server 110 selects one or more frames from the plurality of frames associated with each of the at least two anatomical planes for the uterus, the endometrium, and the other uterine pathologies, where each of the one or more selected optimal frames corresponds to a respective anatomical structure and provides an optimal cross-sectional representation suitable for accurate segmentation and volumetric reconstruction.
[0055] In some embodiments, selection of the optimal frames and detection of uterine pathologies are performed using a detection machine-learning model trained to classify the plurality of frames into frames comprising uterine pathology information and frames without pathological information. In some embodiments, the detection machine-learning model includes a deep-learning-based classification model implemented using a residual neural network (ResNet) architecture.
[0056] The selected optimal frames are provided to a localization machine-learning model configured to automatically generate bounding-box prompts corresponding to regions of interest associated with uterine structures, endometrial structures, and uterine pathologies. In some embodiments, the localization machine-learning model includes an object-detection model implemented using a You-Only-Look-Once (YOLO) architecture, which identifies regions corresponding to optimal cross-sectional representations of uterine pathologies.
[0057] The multimodal image processing and 3D modelling server 110 segments uterine boundaries, endometrial boundaries, and other uterine pathology boundaries within the one or more selected optimal frames using a boundary- segmentation machine-learning model trained using a prompt-based segmentation architecture and a training dataset comprising medical images annotated with uterine boundaries, endometrial boundaries, and uterine pathology boundaries. In some embodiments, the boundary-segmentation machine-learning model includes a MedicalSegment Anything Model (MedSAM) configured to receive bounding-box prompts and generate segmentation masks within the defined regions of interest.
[0058] In some embodiments, bounding boxes for each of the uterus, the endometrium, and the uterine pathology boundaries are automatically generated based on boundary detection performed by the boundary-segmentation machine-learning model.
[0059] The multimodal image processing and 3D modelling server 110 maps the segmented uterine boundaries, the endometrial boundaries, and the other uterine pathology boundaries from the one or more selected optimal frames into a three-dimensional spatial coordinate system based on depth-scale normalization and alignment of the imaging planes. In some embodiments, the multimodal image processing and 3D modelling server 110 automatically extracts a depth scale embedded in each frame and converts the extracted depth scale into real anatomical distances for accurate volumetric mapping and spatial alignment.
[0060] The multimodal image processing and 3D modelling server 110 generates a three-dimensional rendered model of the uterus, the endometrium, and the other uterine pathologies by reconstructing a volumetric representation from the mapped volumetric spatial coordinates. In some embodiments, generating the three-dimensional rendered model includes combining segmented boundaries from the anatomical planes to reconstruct volumetric geometry of the uterus, the endometrium, and the other uterine pathologies.
[0061] The reconstructed three-dimensional rendered anatomical model of the uterus including both uterine structure and an endometrial layer, thereby enabling visualization and assessment of endometrial thickness, size, and accurate spatial characteristics of uterine pathologies.
[0062] In some embodiments, segmented endometrial boundaries are employed by the multimodal image processing and 3D modelling server 110 to generate a virtual hysteroscopy visualization comprising a navigable fly-through of a reconstructed uterine cavity, thereby enabling internal visualization of the uterine cavity without invasive instrumentation.
[0063] The system 100 provides an automated solution for generating a three-dimensional (3D) rendered model of the uterus, the endometrium, and other uterine pathologies directly from two-dimensional ultrasound or magnetic resonance imaging data, offering advantages in processing speed, volumetric accuracy, and reduction of operator dependency. In clinical use, the system 100 automatically detects uterine structures, selects optimal frames from at least twoanatomical planes, segments uterine boundaries, endometrial boundaries, and uterine pathology boundaries using a prompt-based segmentation architecture, and reconstructs an anatomically accurate three-dimensional model that enables precise visualization and spatial assessment of uterine anatomy and uterine pathologies. The system 100 combines automatic optimal-frame selection, depth-scale normalized volumetric mapping across anatomical planes, and integrated three-dimensional reconstruction to generate a complete uterine model using selected imaging planes, thereby delivering a computationally efficient and clinically effective workflow not achieved by systems on manual segmentation or full three-dimensional imaging acquisition.
[0064] FIG. 2 illustrates a block diagram of the 3D uterus model generating server -according to some embodiment herein. The multimodal image processing and 3D modelling server 110 includes a database 202, multimodal image receiving module 204, a frame extraction module 206, an orientation and spatial position determination module 208, an optimal frame selection module 210, a boundary segmentation module 212, a mapping module 214, and a three-dimensional rendering and reconstruction module 216, each communicatively connected to one another to perform automated generation of a three-dimensional rendered model of a uterus with an endometrium and other uterine pathologies.
[0065] The database 202 includes a processor and a memory. The memory stores data and instructions required for execution. The processor executes the instruction, performs computational and logical operations, and runs programs, algorithm, and database-replated processor.
[0066] The multimodal image receiving module 204 is configured to receive multimodal medical images of the uterus, the endometrium, and other uterine pathologies from a plurality of user devices. The multimodal medical images include at least one of two-dimensional ultrasound volume data or two-dimensional magnetic resonance imaging volume data acquired in multiple anatomical planes including sagittal, axial, and coronal planes. The received images are stored in memory and provided as an initial input for subsequent frame extraction and processing.
[0067] The frame extraction module 206 extracts a plurality of frames from each imaging plane of the received multimodal medical images. Each extracted frame represents a discrete two-dimensional anatomical slice preserving uterine structural information, endometrial information, and uterine pathological features. The extracted frames are forwarded for anatomical orientation analysis, spatial position determination, and machine-learning-based detection and segmentation.
[0068] The orientation and spatial position determination module 208 determines an anatomical orientation and spatial position of uterine pathologies with respect to the uterus and the endometrium within the extracted frames. The orientation and spatial position determination module 208 identifies the anatomical plane associated with each frame and determines spatial relationships between anatomical structures across multiple planes. In some embodiments, the orientation and spatial position determination module 208 automatically extracts depth-scale information embedded in each frame and converts the extracted depth scale into real anatomical distances for accurate volumetric mapping and spatial alignment.
[0069] The optimal frame selection module 210 selects one or more optimal frames from the plurality of frames associated with each of at least two anatomical planes for the uterus, the endometrium, and the other uterine pathologies. In some embodiments, selection of optimal frames is performed using a detection machine-learning model implemented as a deep-learning-based residual neural network (ResNet) classifier trained to classify frames into frames comprising uterine pathology information and frames without pathological information. Only frames classified as containing uterine pathology information are selected for subsequent localization and segmentation.
[0070] In some embodiments, a localization machine-learning model implemented using a You-Only-Look-Once (YOLO) object detection architecture is applied to the selected frames. The localization model automatically generates bounding-box prompts corresponding to regions of interest associated with uterine structures, endometrial structures, and uterine pathologies and assigns probability scores to predicted object regions. The bounding-box prompts identify frames and regions corresponding to optimal cross-sectional representations of uterine pathologies.
[0071] The boundary segmentation module 212 segments uterine boundaries, endometrial boundaries, and other uterine pathology boundaries within the one or more selected optimal frames. In some embodiments, the boundary segmentation module 212 employs a boundarysegmentation machine-learning model implemented using a Medical Segment Anything Model (MedSAM) architecture. The selected optimal frame and the corresponding bounding-box prompt are provided as inputs to the MedSAM model, which generates segmentation masks corresponding to uterine boundaries, endometrial boundaries, and other uterine pathology boundaries within the region of interest.
[0072] In some embodiments, the boundary segmentation module 212 automaticallygenerates bounding boxes for each of the uterus, the endometrium, and the uterine pathology boundaries based on boundary detection performed by the boundary-segmentation machinelearning model. The segmented boundary representations are provided for volumetric mapping and three-dimensional reconstruction.
[0073] The mapping module 214 maps the segmented uterine boundaries, the endometrial boundaries, and the other uterine pathology boundaries from the selected optimal frames into a three-dimensional spatial coordinate system based on depth-scale normalization and alignment of the imaging planes. The mapping module 214 establishes accurate spatial correspondence among segmented anatomical structures across multiple planes and generates mapped volumetric spatial coordinates corresponding to each anatomical boundary.
[0074] The three-dimensional rendering and reconstruction module 216 reconstructs a volumetric representation of the uterus, the endometrium, and the other uterine pathologies from the mapped volumetric spatial coordinates. The module combines segmented boundaries from at least two anatomical planes and reconstructs a volumetric geometry of the uterus, the endometrium, and the uterine pathologies, thereby generating a three-dimensional rendered model with accurate dimensional and spatial information.
[0075] In some embodiments, the reconstructed three-dimensional rendered anatomical model of the uterus including both uterine structure and an endometrial region, thereby enabling visualization and assessment of endometrial thickness, uterine size, cavity morphology, and spatial characteristics of uterine pathologies.
[0076] In some embodiments, segmented endometrial boundaries generated by the boundary segmentation module 212 are employed by the three-dimensional rendering and reconstruction module 216 to generate a virtual hysteroscopy visualization comprising a navigable fly-through of a reconstructed uterine cavity, thereby enabling non-invasive internal visualization of the uterine cavity.
[0077] In some embodiments, the multimodal image processing and 3D modelling server 110 provides a manual mode and an artificial intelligence mode. In the manual mode, selection of optimal frames, annotation of anatomical boundaries, and segmentation of uterine structures and uterine pathologies are performed manually by a clinical user. In the artificial intelligence mode, frame selection, bounding-box generation, boundary segmentation, spatial mapping, and volumetric reconstruction are performed automatically using the machine learning pipeline. Auser-selectable toggle is provided to switch between the manual mode and the artificial intelligence mode.
[0078] The multimodal image processing and 3D modelling server 110 integrates multimodal image receiving, frame extraction, orientation and spatial position determination, optimal frame selection, boundary segmentation, spatial mapping, and volumetric reconstruction to generate a complete three-dimensional rendered model from multimodal two-dimensional imaging data. The automated processing reduces manual intervention, improves volumetric accuracy, and enables consistent generation of anatomically accurate three-dimensional uterine models for visualization, assessment, procedural planning, and clinical decision support.
[0079] FIG. 3 illustrates a block diagram of a MINV device 300 utilizing 2D ultrasound volume data or 2D MRI volume data according to some embodiments herein. The MINV device 300 includes a 2D ultrasound volume data 102, a 2D MRI volume data 104, a data transfer device 302, a computing device 304, and a display device 306.
[0080] The 2D ultrasound volume data 102 provides detailed, two-dimensional images for visualizing internal uterine structures, endometrium structures, and other uterine pathologies structures. The data is commonly used for diagnostic assessment monitoring anatomical changes, and supporting accurate medical evaluation of the uterus and associated uterine pathologies. The 2D MRI volume data 104 provides high-resolution, two-dimensional images for detailed visualization of uterine structures endometrium structures, and other uterine pathologies structures, aiding in anatomical assessment and spatial localization.
[0081] In some embodiments, the 2D ultrasound volume data 102 and the 2D MRI volume data 104 can be captured from patients diagnosed with of the uterus, the endometrium, and other uterine pathologies The 2D ultrasound volume data 102 may be stored in standard medical imaging formats such as Digital Imaging and Communications in Medicine (DICOM), and may reside on a Picture Archiving and Communication System (PACS) server, which can be accessed by the MINV device 106 and converted into compatible formats for subsequent analysis. The 2D ultrasound volume data 102 may be stored on external storage devices, such as Universal Serial Bus (USB) drives, for direct access by the MINV device 106. Real-time 2D ultrasound volume data 102 may also be streamed from an ultrasound machine and recorded by the MINV device 106 in a compatible format for immediate processing.
[0082] The 2D ultrasound volume data 102 and the 2D MRI volume data 104 areconnected to the data transfer device 302, which transfers the acquired imaging data to the computing device 304. In some embodiments, the data transfer device 302 includes a frame grabber, a wired or wireless communication module, or any other equipment capable of transferring digital image data from a source device to the computing device 304. The computing device 304 processes the transferred data for detection, segmentation, and 3D reconstruction of the uterus, the endometrium, and the other uterine pathologies.
[0083] The display device 306 is connected to the computing device 304 and is configured to visually present the processed imaging data, including the three-dimensional rendered model of the uterus, the endometrium, and the other uterine pathologies. The display device 306 provides high-resolution of anatomical structures, spatial relationships, and reconstructed volumetric geometry, thereby aiding healthcare professionals in visualization, assessment, and procedural planning. The system 300 including automated data transfer, reduced operator dependency, improved accuracy in lesion visualization, rapid processing of multimodal medical images, and seamless integration with 3D reconstruction workflows, enabling clinicians to make informed decisions efficiently while reducing time and potential errors in clinical assessment.
[0084] FIG. 4 illustrates a 3D-rendered model 400 of a uterus with mapping uterine pathologies according to some embodiments. The MINV device 106 postprocesses the annotated frames and converts the annotated frames into the three-dimensional rendered model 400 of the uterus with other uterine pathologies mapped with accurate dimensional information and spatial locations. The 3D-rendered model 400 includes a first uterine pathologies 402, a second uterine pathologies 404, and a third fibroid 406. The Radio Frequency Ablation (RFA) is another treatment option for uterine pathologies, in which a high-frequency alternating current is applied to a thin needle or electrode. The heat generated by the current spreads to the uterine pathologies tissue, causing necrosis (cell death). The success of the RFA can be confirmed through the MINV device 106, where the uterine pathologies is visualized in the 3D model. When ablation begins, indicated by clicking the “start procedure” button, heat is applied to the fibroid tissue, after the appropriate duration and power are used, the fibroid undergoes necrosis, which is represented by its transition to a whitish colour once the “stop procedure” button is clicked.
[0085] The 3D-rendered model 400 integrates spatial localization of uterine pathologies, real-time visualization of treatment effects, enhanced procedural precision, reduced risk to surrounding tissues, improved patient counselling and understanding, and support for pre-treatment and post-treatment evaluation, all while integrating frame selection, segmentation, annotation, volumetric mapping, and 3D reconstruction to provide a clinically efficient and transformative workflow.
[0086] FIG. 5 illustrates a fly through feature that enables the user to view an inside view 500 of the uterine cavity which enables a virtual hysteroscopy according to some embodiments. The flythrough feature allows the trained clinical professional to navigate through a 3D-rendered environment or model as if they are moving within it.
[0087] The fly-through feature provides a dynamic internal view of the uterine cavity, and other uterine pathologies enabling enhanced visualization, measurement, and analysis. This feature supports pre-procedural planning, allowing clinicians to assess anatomical orientation, lesion location, and spatial relationships within the uterus with improved accuracy. The virtual hysteroscopy serves as a non-invasive alternative to traditional hysteroscopy, generating a high-resolution 3D visualization of the uterine cavity using the previously segmented and annotated frames from multimodal 2D imaging. Clinicians can detect uterine abnormalities without performing invasive procedures, thereby increasing patient comfort and safety.
[0088] The fly-through virtual hysteroscopy feature includes enhanced spatial understanding of uterine structures, improved diagnostic accuracy, non-invasive evaluation, reduced procedural risk, real-time navigation within the 3D model, support for treatment planning and counselling, and integration with pre and post-treatment visualization workflows. By combining 3D reconstruction, segmentation, volumetric cavity generation, and interactive navigation, the system delivers a comprehensive, clinically transformative platform for uterine assessment and procedural planning.
[0089] FIG. 6 are flow diagrams that illustrate a method for generating the three-dimensional (3D) rendered model of the uterus along with uterine structural abnormalities according to some embodiment herein. At a step 602, the method includes receiving multimodal medical images of the uterus, the endometrium, and other uterine pathologies, from a plurality of user devices, where the multimodal medical images include at least one of two-dimensional (2D) ultrasound volume data or 2D magnetic resonance imaging (MRI) volume data. At a step 604, the method includes extracting a plurality of frames from each imaging plane of the multimodal medical images, where the imaging planes include at least two anatomical planes. At a step 606, the method includes determining an anatomical orientation and spatial position of the uterinepathologies with respect to the uterus and the endometrium within the plurality of frames. At a step 608, the method includes selecting one or more frames from the plurality of frames associated with each of the at least two anatomical planes for the uterus, the endometrium, and the other uterine pathologies, where each of the one or more selected optimal frames of respective anatomical structure. At a step 610, the method includes segmenting uterine boundaries, endometrial boundaries, and other uterine pathology boundaries within the one or more selected optimal frames. At a step 612, the method includes mapping the segmented uterine boundaries, the endometrial boundaries, and the other uterine pathology boundaries, from the one or more selected optimal frames into three-dimensional spatial coordinate system based on the depth-scale normalization and alignment of the imaging planes. At a step 614, the method includes generating a 3D rendered model of the uterus, the endometrium, and the other uterine pathologies with accurate dimensional and spatial information for the uterine pathologies by reconstructing a volumetric representation from mapped volumetric spatial coordinates.
[0090] FIG. 7 is representative hardware environment for practicing the embodiments herein, with reference to FIGS. 1 through 6. This schematic drawing illustrates a hardware configuration of a multimodal image processing and 3D modelling server 110 / computer system / computing device in accordance with the embodiments herein. The system includes at least one processing device CPU 10 that may be interconnected via system bus 14 to various devices such as a random-access memory (RAM) 15, read-only memory (ROM) 16, and an input / output (I / O) adapter 17. The I / O adapter 17 can connect to peripheral devices, such as disk units 12 and program storage drives 13 that are readable by the system. The system can read the inventive instructions on the program storage drives 13 and follow these instructions to execute the methodology of the embodiments herein. The system includes a user interface adapter 20 that connects a keyboard 18, mouse 19, speaker 25, microphone 23, and / or other user interface devices such as a touch screen device (not shown) to the bus 14 to gather user input. Additionally, a communication adapter 21 connects the bus 14 to a data processing network 26, and a display adapter 22 connects the bus 14 to a display device 24, which provides a graphical user interface (GUI) 30 of the output data in accordance with the embodiments herein, or which may be embodied as an output device such as a monitor, printer, or transmitter, for example.
[0091] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readilymodify and / or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the scope.
Claims
AMENDED CLAIMSreceived by the International Bureau on 26 June 2026 (26.06.2026)CLAIMSI / We claim:
1. A method for generating a three-dimensional (3D) rendered model of a uterus with an endometrium along with other uterine pathologies, wherein the method comprises:receiving multimodal medical images of the uterus, the endometrium, and other uterine pathologies, from a plurality of user devices, wherein the multimodal medical images comprise at least one of two-dimensional (2D) ultrasound volume data or 2D magnetic resonance imaging (MRI) volume data;extracting a plurality of frames from each imaging plane of the multimodal medical images, wherein the imaging planes comprise at least two anatomical planes;determining an anatomical orientation and spatial position of the uterine pathologies with respect to the uterus and the endometrium within the plurality of frames;executing a detection machine-learning model trained to classify the plurality of frames into frames comprising uterine pathology information and to identify one or more optimal frames from the plurality of frames associated with each of the at least two anatomical planes for the uterus, the endometrium, and the other uterine pathologies, wherein each of the one or more optimal frames corresponds to a cross-sectional representation of a respective anatomical structure;segmenting uterine boundaries, endometrial boundaries, and other uterine pathology boundaries within the one or more optimal frames;automatically extracting a depth scale embedded in each of the one or more optimal frames and converting the extracted depth scale into corresponding real anatomical distances; andmapping the segmented uterine boundaries, the endometrial boundaries, and the other uterine pathology boundaries, from the one or more optimal frames into a three-dimensional spatial coordinate system based on the depth-scale normalization and alignment of the imaging planes and generating the 3D rendered model of the uterus, the endometrium, and the other uterine pathologies with accurate dimensional and spatial information by combining the mapped segmented uterine boundaries, the endometrial boundaries, and the other uterinepathology boundaries from the at least two anatomical planes to reconstruct a volumetric representation from mapped volumetric spatial coordinates.
2. The method as claimed in claim 1, wherein the segmentation of the uterine structures, endometrium structures and uterine pathologies use a boundary-segmentation machinelearning model trained using a prompt-based segmentation architecture and a training dataset comprising medical images annotated with uterine boundaries, endometrial boundaries, and the other uterine pathology boundaries, the boundary-segmentation machine-learning model being configured to segment the uterine boundaries, the endometrial boundaries, and the other uterine pathology boundaries within the regions of interest defined by the bounding-box prompts.
3. The method as claimed in claim 1, wherein the detection machine-learning model comprises a deep-leaming-based classification model implemented using a residual neural network (ResNet) architecture.
4. (currently amended) The method as claimed in claim 1, wherein the localization machinelearning model comprises an object-detection model implemented using a You-Only-Look-Once (YOLO) architecture, the localization machine-learning model configured to automatically generate bounding-box prompts corresponding to regions of interest associated with uterine structures, endometrium structures and the other uterine pathologies within the one or more optimal frames5. The method as claimed in claim 2, wherein the boundary- segmentation machine-learning model comprises a Medical Segment Anything Model (MedSAM), configured to receive the bounding-box prompt corresponding to a detected uterine pathology and to generate segmentation masks for the uterine boundaries, the endometrial boundaries, and the other uterine pathology boundaries within the bounding-box prompt.
6. The method as claimed in claim 1, wherein the three-dimensional rendered anatomical model of the uterus including both a uterine structure and an endometrial layer, thereby enabling visualization and assessment of endometrial thickness, size, and accurate spatial characteristics of uterine pathologies.
7. The method as claimed in claim 1, wherein segmented endometrial boundaries are employed to generate a virtual hysteroscopy visualization comprising a navigable fly-through of a reconstructed uterine cavity.
8. A system for generating a three-dimensional (3D) rendered model of a uterus with an endometrium along with other uterine pathologies, wherein the system comprises:a memory;a processor communicatively connected to the memory and configured to: receive multimodal medical images of the uterus, the endometrium, and other uterine pathologies, from a plurality of user devices, wherein the multimodal medical images comprise at least one of two-dimensional (2D) ultrasound volume data or 2D magnetic resonance imaging (MRI) volume data;extract a plurality of frames from each imaging plane of the multimodal medical images, wherein the imaging planes comprise at least two anatomical planes;determine an anatomical orientation and spatial position of the uterine pathologies with respect to the uterus and the endometrium within the plurality of frames;execute a detection machine-learning model trained to classify the plurality of frames into frames comprising uterine pathology information and to identify one or more optimal frames from the plurality of frames associated with each of the at least two anatomical planes for the uterus, the endometrium, and the other uterine pathologies, wherein each of the one or more optimal frames corresponds to a cross-sectional representation of a respective anatomical structure;segment uterine boundaries, endometrial boundaries, and other uterine pathology boundaries within the one or more optimal frames;automatically extract a depth scale embedded in each of the one or more optimal frames and converting the extracted depth scale into corresponding real anatomical distances; and map the segmented uterine boundaries, the endometrial boundaries, and the other uterine pathology boundaries, from the one or more optimal frames into a three-dimensional spatial coordinate system based on the depth-scale normalization and alignment of the imaging planes and generate the 3D rendered model of the uterus, the endometrium, and the other uterine pathologies with accurate dimensional and spatial information by combining the mapped segmented uterine boundaries, the endometrial boundaries, and the other uterine pathology boundaries from the at least two anatomical planes to reconstruct a volumetric representation from mapped volumetric spatial coordinates.