A method for determining the likelihood of endometriosis

Abstract

Broadly speaking, the disclosure describes a computer-implemented method for detecting at least one sign of endometriosis, such as Pouch of Douglas (POD) obliteration, with a solution to the missing modality problem Beneficially, the methods of the present techniques can use either TVUS or MRI data to classify at least one sign of endometriosis accurately and thus indicate the diagnosis of endometriosis of a patient.

Description

A method for determining the likelihood of endometriosis Technical Field

[0001] The present invention relates to computer-implemented methods and apparatuses for analysing images and video clips to detect at least one sign of endometriosis and thus indicate a probability of a diagnosis of endometriosis. Background of Invention

[0002] Endometriosis is a common gynaecological condition that affects approximately 10% of women, girls and those assigned female at birth. Pathologically, endometriosis refers to endometrial-like tissue growing outside of the uterus, resulting in pain, prolonged menstrual bleeding and infertility. The current therapeutic strategies do not offer a cure for endometriosis and are primarily aimed at mitigating the clinical symptoms. Therefore, an accurate early diagnosis of endometriosis is critical to develop preventative treatment strategies and early intervention to mitigate disease progression.

[0003] Visualisation of endometriosis during laparoscopy has been the only method of diagnosis. This is costly, carries risks and has limited accessibility given the high prevalence of endometriosis. It takes on average 6.4 years to obtain a diagnosis in Australia, with many people with endometriosis having a compromised quality of life and having the reality of their symptoms questioned by medical professionals and the community. Those with endometriosis have a higher use of healthcare resources in the 10 years prior to diagnosis than after they have been diagnosed. A cost-effective non-invasive diagnostic test would reduce diagnostic delay and time to initiation of targeted appropriate treatment, reducing healthcare costs and potential need for surgery.

[0004] Menstrual fluid and shed endometrial tissue that flows retrogradely through the fallopian tubes accumulates in the Pouch of Douglas in most women and those assigned female at birth. In approximately 10%, endometrial tissue engrafts creating inflammatory endometriosis lesions that grow and shed in response to the menstrual cycles reproductive hormone changes endometriosis (Sampson’s Theory). This triggers an inflammatory response, painful periods, chronic pelvic pain, complex pain syndromes and infertility.

[0005] One way of diagnosing endometriosis is to detect Pouch of Douglas (POD) obliteration. The obliteration of the POD is a strong diagnostic marker for endometriosis amongst others. The POD (also known as the rectouterine pouch or the posterior cul-de-sac), is a small space located between the rectum and the back wall of the uterus.

[0006] Obliteration of the POD can be visualised using Magnetic Resonance Imaging (MRI) and / or Transvaginal Ultrasound (TVUS) scanning. Our recent results suggest that the classification of POD obliteration is more accurate with TVUS than with MRI.

[0007] In MRI, POD obliteration can manifest as pelvic nodules or plaque-like lesions and adhesions that are visible on T2-weighted images (see positive POD obliteration in Figure 1C compared to a negative case in Figure 1A).

[0008] Using TVUS, a dynamic “sliding sign” technique can identify POD obliteration when the cervix is gently pressed. If the anterior rectum slides freely on the uterus there is a positive classification which is a normal finding (Figure 1B). If the movement is limited or restricted, there is a negative classification which shows an obliterated POD ( Figure 1D).

[0009] The expertise required to diagnose endometriosis from TVUS and MRI is not widely available in clinical practice. The diagnostic accuracy of POD obliteration using TVUS is between 75% and 95%, whereas for MRI, manual accuracy (endometriosis detection by a human expert) is between 40% and 69%.

[0010] In current practice, patients typically undergo only one of these two examinations (either MRI or TVUS) and particularly with TVUS there is variability in the ability of sonographers to elicit the sliding sign. Because of the relatively low accuracy of MRI and TVUS approaches when used independently, there is a need for a better alternative diagnostic tool, capable of producing a diagnosis with higher independent or at least a similar accuracy.

[0011] Any reference herein to a patent document or other matter which is given as prior art is not to be taken as an admission that the document or matter was known or that the information it contains was part of the common general knowledge as at the priority date of any of the claims. Summary of Invention

[0012] According to the present invention, there is provided an apparatus, a method, a computer program, and a computer-readable medium as set forth in the appended claims. Other features of the invention will be apparent from the dependent claims, and the description which follows.

[0013] In one aspect, there is provided a computer-implemented method for determining the likelihood of endometriosis, the method comprising: obtaining an MRI result or TVUS result of a subject; inputting the MRI result or TVUS result into an artificial intelligence (AI) model and / or a Learnable Cross-Modal Knowledge Distillation (LCKD) model; and detecting, by theAI model and / or the LCKD model, at least one sign of endometriosis ; and outputting the result of the at least one sign of endometriosis detection; wherein the at least one sign of endometriosis is at least one of: Pouch of Douglas (POD) obliteration, bowel nodules, ovarian endometrioma and uterosacral ligament (USL) nodules. Outputting the result of the at least one sign of endometriosis detection (e.g. POD detection) results in determining the likelihood of endometriosis. The skilled person will appreciate that the method may comprise obtaining an MRI result and / or TVUS result of a subject; inputting the MRI result and / or TVUS result into an artificial intelligence (AI) model and / or a Learnable Cross-Modal Knowledge Distillation (LCKD) model. Thus, both imaging modalities may be used in the method.

[0014] The AI model may comprise: extracting, by a magnetic resonance imaging (MRI) feature extractor, at least one MRI embedding from an endometriosis MRI dataset in an MRI stream; extracting, by a transvaginal ultrasound (TVUS) feature extractor, at least one TVUS embedding from a TVUS dataset in a TVUS stream; measuring uncertainty in the MRI stream; measuring uncertainty in the TVUS stream; cross-modally fusing the MRI and TVUS embeddings by multiplying the at least one MRI embedding by the measured uncertainty in the TVUS stream and multiplying the at least one TVUS embedding by the measured uncertainty in the MRI stream; concatenating the cross-modally fused MRI and TVUS embeddings; and detecting the at least one sign of endometriosis.

[0015] The AI model may further comprise: pre-training the MRI feature extractor based on an MRI dataset; and / or pre-training TVUS feature extractor based on an TVUS dataset.

[0016] The AI model may further comprise: matching MRI images with positive / negative POD obliteration labels of an endometriosis MRI dataset to TVUS videos of the TVUS dataset with a negative / positive sliding sign. The AI model may further comprise: matching MRI images with positive / negative sign of endometriosis labels of an endometriosis MRI dataset to TVUS videos of the TVUS dataset with a negative / positive sign of endometriosis.

[0017] The AI model may comprise: passing the concatenated cross-modally fused MRI and TVUS embeddings to a linear layer for prediction.

[0018] The endometriosis MRI dataset and the TVUS dataset may be unpaired for optimising the model.

[0019] The endometriosis MRI dataset may comprise a plurality of MRI images, wherein at least one of the plurality of MRI images may comprise a labelled POD obliteration case. The endometriosis MRI dataset may comprise a plurality of MRI images, wherein at least one of the plurality of MRI images comprises at least one of: a labelled POD obliteration case, a labelledbowel nodule case, a labelled ovarian endometrioma case and a labelled uterosacral ligament nodule case.

[0020] The TVUS dataset may comprise a plurality of sliding sign TVUS video clips, wherein at least one of the plurality of the TVUS video clips may comprise a labelled POD obliteration case. The TVUS dataset may comprise a plurality of sliding sign TVUS video clips, wherein at least one of the plurality of the TVUS video clips comprises at least one of: a labelled POD obliteration case, a labelled bowel nodule case, a labelled ovarian endometrioma case and a labelled uterosacral ligament nodule case.

[0021] The MRI dataset may be unlabelled and comprises a plurality of MRI images of a female pelvic region.

[0022] The AI model may comprise: measuring channel-wise uncertainties in the MRI stream by a first random network prediction (RNP) module; and / or measuring channel-wise uncertainties in the TVUS stream by measuring uncertainty in the MRI stream by a second RNP module.

[0023] Both the first and the second RNP modules may comprise a fixed-weight random network and a learnable prediction network.

[0024] The AI model may comprise: training the first and the second RNP modules based on minimisation of a loss function computed from the difference between outputs by the prediction and random networks.

[0025] The AI model may comprise: outputting, by the first RNP module, a single scalar per modality to represent the uncertainty; and outputting, by the second RNP module, a single scalar per modality to represent the uncertainty.

[0026] Obtaining the MRI result or the TVUS result of the subject may comprise receiving the MRI result and / or TVUS result by a medical professional or the subject.

[0027] The MRI result (e.g. MRI data) may be an MRI image. The TVUS result (e.g. TVUS data) may be a sliding sign TVUS video clip.

[0028] The TVUS feature extractor may be a convolutional neural network.

[0029] The MRI feature extractor may be 3D vision transformer.

[0030] The detection of at least one of: POD obliteration, a bowel nodule, an ovarian endometrioma and an uterosacral ligament nodule may be used for diagnosis of endometriosis.

[0031] The LCKD model may comprise electing a teacher modality by a validation process.

[0032] The LCKD model may comprise distilling cross-modal knowledge between each pair of available teacher and student modalities by a loss function.

[0033] The LCKD model may comprise training the teacher and student modality pair, wherein during training the student modality mimics behaviour and predictions of the teacher modality.

[0034] In another aspect, there is provided a data processing device comprising means for carrying out the method described above.

[0035] In another aspect, there is provided a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method described above.

[0036] In another aspect, there is provided a computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method described above.

[0037] As will be appreciated by one skilled in the art, the present techniques may be embodied as a method, device, computer program product or a computer-readable storage medium. Accordingly, present techniques may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0038] Furthermore, the present techniques may take the form of a computer program product embodied in a computer-readable medium having computer-readable program code embodied thereon. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium may be, for example, but is not limited to, any physical device or material that can store digital data, such as a hard disk, CD-ROM, flash drive, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.

[0039] Computer program code for carrying out operations of the present techniques may be written in any combination of one or more programming languages, including object-oriented programming languages and conventional procedural programming languages. Code components may be embodied as procedures, methods or the like, and may comprise sub- components which may take the form of instructions or sequences of instructions at any of thelevels of abstraction, from the direct machine instructions of a native instruction set to high- level compiled or interpreted language constructs.

[0040] Examples of the present techniques also provide a non-transitory data carrier carrying code which, when implemented on a processor, causes the processor to carry out any of the methods described herein.

[0041] The techniques may provide processor control code to implement the above- described methods, for example on a general-purpose computer system or on a digital signal processor (DSP). The techniques also provide a carrier-carrying processor control code to, when running, implement any of the above methods, in particular on a non-transitory data carrier. The code may be provided on a carrier such as a disk, a microprocessor, CD- or DVD- ROM, programmed memory such as non-volatile memory (e.g. Flash) or read-only memory (firmware), or on a data carrier such as an optical or electrical signal carrier. Code (and / or data) to implement embodiments of the techniques described herein may comprise source, object or executable code in a conventional programming language (interpreted or compiled) such as Python, C, or assembly code, code for setting up or controlling an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), or code for a hardware description language such as Verilog (RTM) or VHDL (Very high speed integrated circuit Hardware Description Language). As the skilled person will appreciate, such code and / or data may be distributed between a plurality of coupled components in communication with one another. The techniques may comprise a controller which includes a microprocessor, working memory and program memory coupled to one or more of the components of the device.

[0042] It will also be clear to one of skill in the art that all or part of a logical method according to embodiments of the present techniques may suitably be embodied in a logic apparatus comprising logic elements to perform the steps of the above-described methods, and that such logic elements may comprise components such as logic gates in, for example a programmable logic array or application-specific integrated circuit. Such a logic arrangement may further be embodied in enabling elements for temporarily or permanently establishing logic structures in such an array or circuit using, for example, a virtual hardware descriptor language, which may be stored and transmitted using fixed or transmittable carrier media.

[0043] The present techniques may be realised in the form of a data carrier having functional data thereon, said functional data comprising functional computer data structures to, when loaded into a computer system or network and operated upon thereby, enable said computer system to perform all the steps of the above-described method.

[0044] The methods described above may be wholly or partly performed on an apparatus, i.e. an electronic device. The method may be processed by an artificial intelligence-dedicated processor designed in a hardware structure specified for artificial intelligence model processing. The artificial intelligence model may be obtained by training. Here, "obtained by training" means that a predefined operation rule or artificial intelligence model configured to perform a desired feature (or purpose) is obtained by training a basic artificial intelligence model with multiple pieces of training data by a training algorithm. The artificial intelligence model may include a plurality of neural network layers.

[0045] As mentioned above, the present techniques may be implemented using an AI model. A function associated with AI may be performed through the non-volatile memory, the volatile memory, and the processor. The processor may include one or a plurality of processors. The one or a plurality of processors control the processing of the input data in accordance with a predefined operating rule or artificial intelligence (AI) model stored in the non-volatile memory and the volatile memory. The predefined operating rule or artificial intelligence model is provided through training or learning. Here, being provided through learning means that, by applying a learning algorithm to a plurality of learning data, a predefined operating rule or AI model of a desired characteristic is made. The learning may be performed in a device itself in which AI according to an embodiment is performed, and / or may be implemented through a separate server / system. Brief Description of Drawings

[0046] Figure 1A shows an example of an MRI image with negative POD obliteration (normal);

[0047] Figure 1B shows an example of a TVUS image with a positive sliding sign (normal);

[0048] Figure 1C shows an example of an MRI image with positive POD obliteration;

[0049] Figure 1D shows an example of a TVUS image with a negative sliding sign;

[0050] Figure 2A shows a schematic diagram representing the MRI feature extractor pre- training;

[0051] Figure 2B shows a schematic diagram representing the TVUS feature extractor pre- training;

[0052] Figure 2C shows a schematic diagram representing multimodal fusion with uncertainty estimation;

[0053] Figure 3 shows a flowchart of the method representing obtaining an AI model according to the present techniques;

[0054] Figure 4 shows a schematic diagram representing an exemplary Learnable Cross- Modal Knowledge Distillation (LCKD) model;

[0055] Figure 5 shows a flowchart of the method of obtaining the LCKD model;

[0056] Figure 6 shows the method of detecting at least one sign of endometriosis according to the present techniques;

[0057] Figure 7 shows the results of a case study applying techniques of the present disclosure; and

[0058] Figure 8 shows a schematic illustration of a computing device suitable for the implementation of the methods of the present techniques. Detailed Description

[0059] Broadly speaking, the present disclosure describes a computer-implemented method for endometriosis diagnosis with a solution to the missing modality problem by fusing the TVUS and MRI modalities with estimated uncertainty using an unpaired dataset. Beneficially, the methods of the present techniques can use either TVUS or MRI data to classify at least one sign of endometriosis, e.g. POD obliteration, accurately and thus indicate the diagnosis of endometriosis of a patient. The classification may be binary (yes / no).

[0060] The applicant recognizes that for the diagnosis of endometriosis based on a sign of endometriosis, such as POD obliteration detection, a multimodal analysis combining TVUS and MRI is preferable. MRI provides spatial information and allows precise characterisation of tissues, but cannot capture dynamic information like TVUS. Thus, combining MRI and TVUS permits the identification of distinct and complementary markers, which can improve the diagnostic accuracy for a sign of endometriosis and thus indicate the likelihood of endometriosis. However, patients typically have only either MRI results or TVUS results.

[0061] Broadly speaking, the present disclosure describes an artificial intelligence (AI) model that is trained with unpaired MRI and TVUS data, and preferably tested with just one of the modalities. In some embodiments, the accuracy of results obtained with the AI model using either MRI or TVUS results may be as high as using the best of the two modalities (e.g., TVUS) alone. The present bi-modal AI model is trained to achieve this goal in two stages: 1) pre- training a TVUS feature extractor for the TVUS stream to detect a sign of endometriosis, suchas POD obliteration, from TVUS data and an MRI feature extractor for the MRI stream with a 3D masked auto-encoder (MAE) using a large scale set of unlabelled pelvic 3D MRI volumes; and 2) combining the MRI and TVUS modalities by multimodal fusion with uncertainty estimation of each modality. The skilled person will appreciate that an LCKD model may be used to improve the accuracy of the present method.

[0062] Beneficially, the present techniques provide a multimodal model that is trained to combine the knowledge from TVUS and MRI using unpaired data and thus detect a sign of endometriosis (e.g. POD obliteration) with missing modality. Embodiments of the present techniques use a multimodal learning algorithm based on the Cross-modal Random Network Prediction that is adapted from a multi-modal segmentation that requires all input modalities for testing to a multi-modal POD obliteration classification that handles missing modalities for testing. In the Cross-modal Random Network Prediction, this is achieved by introducing a channel-wise uncertainty estimation to encourage the model to focus more on one modality if the features from the other modality are uncertain. A further benefit of the method of the present technique is that a diagnosis of endometriosis may be achieved non-invasively, i.e. without requiring surgery or anaesthesia.

[0063] Embodiments of the present techniques utilize a multimodal learning algorithm, Cross-modal Random Network Prediction, CRNP, (referred to hereafter as the AI model), that adapts from a multi-modal segmentation that requires all input modalities for testing to a multi- modal sign of endometriosis (e.g. POD obliteration) classification that handles missing modality for testing. It introduces channel-wise uncertainty estimation to prioritise one modality when the other is uncertain. In other examples, the LCKD model may be combined with the Cross-modal Random Network Prediction. The skilled person will appreciate that both the LCKD model and Cross-modal Random Network Prediction model are artificial intelligence models, but for simplicity, the Cross-modal random network prediction model is referred to as “AI model” and the LCKD model is referred to as “LCKD model”.

[0064] The skilled person will appreciate that the multimodal detection of a sign of endometriosis (e.g. POD obliteration) of the present techniques is important because in general, multimodal approaches can achieve better performance than their single-modality counterparts.

[0065] The skilled person will also appreciate that using POD obliteration is one way / sign of diagnosing endometriosis. There may be other signs that can be used to determine endometriosis. The skilled person will appreciate that the methods and systems of the present disclosure may be modified to include the other signs to determine endometriosis.

[0066] The signs of endometriosis may be at least one of: POD obliteration, bowel nodules, ovarian endometrioma and uterosacral ligament (USL) nodules. In other words, the signs of endometriosis may be a POD obliteration and / or bowel nodules and / or ovarian endometrioma and / or uterosacral ligament (USL) nodules. All of these signs of endometriosis may be identified by TVUS and MRI. Thus, the AI model and the LCKD model of the present techniques may be trained to detect at least one sign of endometriosis.

[0067] Bowel nodules, in the context of endometriosis, refer to areas where endometrial- like tissue grows on or within the walls of the bowel, forming small lumps or masses. In TVUS, bowel nodules may be darker (hypoechoic) plaques that can be seen thickening or invading the serosal layers of the sigmoid, colon or rectum or mushrooming into the bowel lumen. On MRI images, the bowel nodules may be thickened or retracted by fibrosis. Multifocal lesions (within 2 cm of the main lesion) can be distinguished from multifocal bowel endometriosis (more than 2 cm apart) which is important in planning surgical excision.

[0068] Beneficially, the detection of endometriosis using the bowel nodules can explain symptoms such as bowel pain, constipation, diarrhoea, nausea and anorexia. Moreover, it can also reduce wasted time and resources spent exploring the gastrointestinal system using colonoscopies and endoscopies by gastroenterologists when a gynaecological referral is required. Further, misdiagnosis of irritable bowel syndrome can be avoided and targeted endometriosis therapies commenced in a timely way.

[0069] Ovarian endometriomas also known as ‘chocolate cysts’ are a specific type of cyst that usually forms on the ovaries or between the ovary and the peritoneal side wall as a result of repeated growth and break down of endometriosis tissue. These cysts are usually filled with old, dark blood. TVUS identifies endometriomas with high sensitivity and specificity by detecting them as cysts with a homogeneous appearance and diffuse, low-level echoes. Similarly, MRI identifies endometriomas with high accuracy as ovarian cysts with hyperintensity (bright white) on T1-weighted images and heterogeneous low to moderate intensities (T2 shading) on T2- weighted images.

[0070] Beneficially, the detection of endometriosis using ovarian endometriomas permits consideration of egg freezing. This is particularly relevant before surgery as surgical excision of endometriomas can reduce egg numbers.

[0071] The uterosacral ligament nodules (behind the uterus) and / or the uterine tortinus (the attachment point of these ligaments at the back of the lower uterus) are some of the most common sites of endometriosis. On MRI images, uterosacral ligament nodules are usually hypointense on T2 weighted images with nodular lesions seen although suspicion is raised ifthe ligaments are thickened or retracted. On T1 weighted images hyperintense, white nodules can be identified. The sensitivity of USL nodule detection by MRI is lower than TVUS.

[0072] Large nodules on the USL ligaments can compress the ureters and cause obstructive renal failure. These nodules can also create inflammation impacting bowel function and fertility. Therefore, beneficially, the detection of endometriosis using uterosacral ligament nodules may influence decisions about medically suppressing periods, undertaking surgery and which symptoms patients should look out for and monitor e.g. kidney and bladder pain and repeat urinary tract infections. Creating an AI model for detecting at least one sign of endometriosis (e.g. POD obliteration) and endometriosis determination

[0073] Figure 2A shows a schematic diagram representing an MRI feature extractor pre- training. Figure 2B shows a schematic diagram representing a TVUS feature extractor pre- training. Figure 2C shows a schematic diagram representing multimodal fusion with uncertainty estimation. Figure 3 shows a flowchart of the method of obtaining an AI model according to the present techniques. The method of creating the AI model is schematically depicted in Figures 2A, 2B and 2C. The AI model performs at least the steps of the method depicted in Figure 3. Figure 6 shows the method of detecting at least one sign of endometriosis (e.g. POD obliteration) which incorporates the AI model of the present techniques.

[0074] The methods of the present techniques 300, 400, 500 are computer-implemented. The methods 300, 400, 500 are suitable for detecting at least one sign of endometriosis (e.g. POD obliteration and / or bowel nodules and / or ovarian endometrioma and / or uterosacral ligament (USL) nodules) and thus outputting a score indicating the probability of the endometriosis diagnosis. The score may be a yes or no determination of endometriosis, or a probabilistic score. Beneficially, the methods 300, 400, 500 can perform this detection outside of the patient’s body. The methods 300, 400, 500 may be implemented by a computer processor, such as the one shown in Figure 8, which may be incorporated into a diagnostic device used in the clinical environment, such as TVUS or MRI console.

[0075] Briefly, the AI model which is created by method 300 consists of two main stages, namely: 1) self-supervised pre-training of the MRI feature extractor on an unlabelled MRI dataset, and training a sliding sign classifier with a TVUS dataset (Figure 2(A),(B)); and 2) fine- tuning the TVUS and MRI feature extractors with a two-stream cross-modal network using unpaired TVUS and MRI data to produce an output being a probabilistic indicator of a sign of endometriosis (e.g. POD obliteration and / or bowel nodules and / or ovarian endometrioma and / or uterosacral ligament (USL) nodules) that is indicative of endometriosis diagnosis (Figure2(C)). As depicted in Figure 2(C), the method of the present techniques uses a bi-modal model that estimates modality uncertainty to allow the prioritisation of one modality when the representation of the other modality has high uncertainty.

[0076] The AI model and the LCKD model are illustrated as being trained on the datasets where the sign of endometriosis is POD obliteration. However, it will be appreciated that the AI model and the LCKD model may be trained (additionally or exclusively) on other signs of endometriosis. The signs of endometriosis may be at least one of: POD obliteration, bowel nodules, ovarian endometrioma and uterosacral ligament (USL) nodules. All of these signs of endometriosis may be identified by TVUS and MRI. Thus, the AI model and the LCKD model of the present techniques may be trained to detect at least one sign of endometriosis.

[0077] In some examples, the AI model and / or the LCKD model may trained on the first sign of endometriosis (e.g. POD obliteration). Then, the AI model and / or the LCKD model may trained on the second sign of endometriosis (e.g. bowel nodules) etc. In some examples, the AI model and / or the LCKD model may be trained simultaneously on two or more signs of endometriosis (e.g. POD obliteration and bowel nodules) as described in more detail below.

[0078] denotes the endometriosis MRI dataset, with N T2 SPACE MRI images ^^^^^^ ^^^^⊂ ^^^^× ^^× ^^, where (H, W and D are height, width and depth) annotated with N one-hot vectors ^^ ^^{0, 1}2denoting the POD obliteration labels. The MRI endometriosis dataset comprises a plurality of T2 SPACE MRI and / or T1 SPACE MRI images. The T2 SPACE MRI images may be pelvic scan images of females. The MRI endometriosis dataset comprises a plurality of MRI images with identified POD obliteration cases. The skilled person will appreciate that it is preferable when the endometriosis MRI dataset images focus on the diagnostic region of POD obliteration near the uterus. The endometriosis MRI dataset comprises 3D images. The endometriosis MRI dataset may be labelled by human annotators.

[0079] denotes the TVUS dataset. The TVUS dataset comprises a plurality of sliding sign TVUS results comprising M video clips ^^^^^^ ^^^^⊂ ^^^^ ^^× ^^ ^^× ^^ ^^containing ^^^^frames of size ^^^^× ^^^^and corresponding sliding sign one-hot label ^^^^^^ {0, 1}2. The TVUS dataset comprises a plurality of “sliding sign” video clips. The sliding sign video clips comprise a plurality of negative sliding sign (POD obliterated) cases. Each sliding sign video comprises at least one frame. Each frame may be labelled. The TVUS dataset may be labelled by human annotators.

[0080] ^^^^represents an MRI dataset for training. The MRI dataset comprises a plurality of MRI scan images for self-supervised pre-training, containing K T2 MRI volumes ^^^^^^ ^^^^, where K>> N and K>>M. The MRI dataset for self-supervised pre-training may comprise T2 MRI volumes from females with endometriosis-like symptoms. However, the skilled person will appreciate that ^^^^dataset may comprise at least one MRI image which may contain signs of other medical conditions and the examination area is not necessarily aligned with the diagnostic area of endometriosis. The MRI dataset ^^^^for self-supervised pre-training is unlabelled. Preferably, the MRI dataset comprises at least five thousand images.

[0081] The endometriosis MRI dataset and the sliding sign TVUS dataset are unpaired. TVUS and endometriosis MRI datasets are matched only based on their POD obliteration label. Accordingly, the method of the present techniques 300 comprises a step of building an unpaired dataset. In particular, the unpaired dataset is built by matching each MRI volumewith a randomly selected TVUS video^^^^, where ^^^^^^= ^^^^^^. In other words, MRI volumes with positive / negative POD obliteration are paired with TVUS videos with a negative / positive sliding sign.

[0082] Even though it is not shown in the Figures, the method 300 may comprise obtaining the endometriosis MRI dataset. The method 300 may further comprise obtaining the TVUS dataset. The method 300 may further comprise obtaining the MRI dataset. The datasets can be obtained from one or more third-party providers. The AI model (alongside the LCKD model) may be pre-trained and trained outside of a computing device in which the AI model is implemented. For instance, a user may download and / or install the AI model on a user’s computing device or on a TVUS or MRI console. Alternatively, all steps of the methods 300, 400, 500 may be implemented on the same computing device or TVUS / MRI console. For instance, the AI model may be incorporated into a software product which can be distributed to an end user device. The AI model may be also saved in a cloud. An end-user may access the AI model in the cloud to upload the user’s TVUS or MRI results to the AI model which provides an indicator of the likelihood of TVUS or MRI results indicating a sign of endometriosis such as POD obliteration which is an indicator of endometriosis diagnosis. At least all of the above- mentioned configurations applicable to the AI model are also applicable to the LCKD model.

[0083] In the pre-training stage, the method of the present techniques 300 may use the dataset ^^^^to optimise the masked reconstruction of MRI volumes and the dataset ^^^^to optimise the sliding-sign classification of the TVUS sequences.

[0084] The method 300 comprises S302 pre-training the MRI feature extractor.

[0085] For the MRI feature extractor pre-training, method 300 extends the self-supervised masked auto-encoder (MAE) to 3D and trains it with the MRI dataset. The MRI dataset is unlabelled T2 MRI dataset ^^^^. Masked autoencoders (MAE) are a type of self-supervised learning method for computer vision that mask random patches of the input image and reconstruct the missing pixels. Beneficially, masked autoencoders are scalable, efficient and effective, and can achieve state-of-the-art results on various image classification and transfer learning tasks.

[0086] The 3D-MAE network may be a 3D Vision Transformer (3D ViT) that is optimised to reconstruct masked MRI volumes through an asymmetric encoder-decoder architecture as shown in Fig.2A. More specifically, each 3D MRI image is divided into 8×8×8 non-overlapping cubes, and a random subset of volume cubes (e.g.50%) is masked out. The skilled person will appreciate that each 3D MRI image may be divided into different dimensions of non- overlapping cubes. The skilled person will also appreciate that other similar transformers may be used. For instance, a Swin Transformer may be used.

[0087] Then, the remaining cubes are fed into the encoder ^^^^ ^^: ^^^^^ ^^ . For example, the encoder may be a 3D ViT with 12 transformer blocks, to be embedded by a linear projection of dimension E = 768 with added positional embeddings. The skilled person will appreciate that the encoder may have more or less than 12 transformer blocks and that the linear projection of dimension E may be more or less than 768. These specific parameters may be chosen or fine tuned based on available GPUs and / or the sizes of the datasets.

[0088] Then, a shallow / lightweight decoder^^ ^ ^^^^, represented, for example, by a 3D ViT with 8 transformer blocks, is applied to the visible tokens from the encoder together with learnable mask tokens to reconstruct the volume. The encoder and decoder are optimised by minimising the mean squared error (MSE) between the reconstructed and original volumes in the voxel space, as follows:(Equation 1)

[0089] where K refers to the size of the unlabelled MRI dataset, L denotes the total number of masked cubes,( ^^) denotes the voxel values of the ^^^^ℎmasked cube in the original volume, while ^^^^^^( ^^) denotes the corresponding ^^^^ℎvoxel values of the reconstructed volume produced by ^^^^^^= ^^^^ ^^( ^^^^ ^^( ^^^^^^)) . After pre-training, the encoder is utilized as an MRI feature extractorto produce the embedding ^^^^= ^^^^ ^^(^^^^)for the downstream POD obliteration classification task.

[0090] The 3D MAE network may be trained from scratch on the ^^^^MRI dataset.

[0091] It will be appreciated that S302 pre-training the MRI feature extractor may further comprise fine-tuning the MRI feature extractor for a region of interest (ROI).

[0092] In the detection of the signs of endometriosis, using MRI, clinicians typically focus on the areas surrounding the uterus within the pelvic cavity. Therefore, the region of interest may be uterus segmentation. However, standard pelvic scans often include extrapelvic regions such as bones, muscles, and adipose tissues. While these additional structures provide valuable context for a full anatomical assessment, they are not directly pertinent to endometriosis and may add unnecessary noise and complexity to the imaging data. Consequently, cropping out irrelevant areas from the original MRI volume enables the model to concentrate on the most critical regions, thereby, beneficially, lowering computational demands and improving the efficiency and accuracy of diagnostic tools.

[0093] In order to fine-tune the MRI feature extractor for a region of interest, Vision Transformer (ViT) models that have been pretrained with a Masked Autoencoder (MAE) by integrating a segmentation head after the encoder, which is defined asand also adding a classification head, defined as ^^^^ ^^: ^^ ^△.

[0094] The optimization of the segmentation model during training may be achieved by simultaneously minimizing both the classification and segmentation loss functions, as outlined below. (Equation 1B)

[0095] where ℓ^^ ^^(. ) denotes the cross-entropy loss, ℓ^^ ^^ ^^(. ) stands for the binary cross- entropy loss, Ω represents the image grid, with ^^^^( ^^) indicating the segmentation label at image position ^^, and ^^^^^^^^ signifies the segmentation prediction at ^^.

[0096] For example, the ROI for detecting both POD obliteration and bowel nodules are situated behind the uterus and within the rectal area. Therefore, the ROI may be defined by applying a pre-determined padding around the predicted uterus segmentation mask. The padding may be empirically established based on the maximum distance from the front of theuterus to the back of the rectum, along with the maximum distance between the coccyx and the vaginal introitus.

[0097] The method 300 comprises S304 pre-training TVUS feature extractor.

[0098] The TVUS feature extractor ^^θ ^^: ^^^^^ △ ( ^^ ^^ ^^ℎ △ ⊂[0, 1]2being the probability simplex) is represented in this example by a convolutional neural network (CNN). For example, the CNN may be the ResNet (2+1) D. In an example, ResNet (2+1) D may contain 18 R(2+1) convolutional layers with batch normalization after convolution and before the Rectified Linear Unit (ReLU) activation. In some embodiments, the ResNet (2+1) D may be pre-trained on the Kinetics-400 dataset and trained for multiple epochs on the TVUS dataset.

[0099] The TVUS feature extractor may be pre-trained to minimise the CE loss (“cross- entropy loss”) with respect to the sliding sign annotation in the sliding sign TVUS dataset ^^^^, as in: (Equation 2)

[0100] After this sample is extracted from the Adaptivelayers, with ^^θ^^^^: ^^^^^ ^^ to form the vector ^^^^. Thus, a feature extractor for the TVUS stream produces at least one TVUS embedding for the downstream POD obliteration classification task. The feature extractor for the TVUS stream may be a CNN.

[0101] Adaptive Average Pooling may be preferred over traditional poolingfor image classification tasks, as it can reduce overfitting and improve model generalization.

[0102] The method 300 comprises S306 building an unpaired dataset. In particular, the unpaired dataset is built by matching each MRI volume ( ^^^^^^, ^^^^^^) ^^ ^^^^with a randomly selected TVUS video ( ^^^^^^, ^^^^^^) ^^ ^^^^, where ^^^^^^= ^^^^^^. In other words, in S306, MRI volumes / images with positive / negative POD obliteration are paired with TVUS videos with a corresponding negative / positive sliding sign. Building an unpaired dataset may comprise matching MRI images with positive / negative POD obliteration labels MRI dataset to randomly selected TVUS videos with asliding sign of the TVUShelp of the TVUS dataset, it is possible to achieve a better POD classification performance on this endometriosis MRI dataset (or viceversa). In more general terms, in S306, MRI volumes / images with a positive / negative sign of endometriosis are paired with TVUS videos with a corresponding positive / negative sign of endometriosis. Building an unpaired dataset may comprise matching MRI images with positive / negative signs of endometriosis labels of the endometriosis MRI dataset to randomly selected TVUS videos with a corresponding negative / positive sign of endometriosis of the TVUS dataset.

[0103] It will be appreciated that in some examples, more than one sign of endometriosis may be combined and used during training. In other words, S306 building an unpaired dataset may comprise a multi-label mix-up. For example, the training data may comprise POD obliteration MRI images and TVUS clips (as described above) and also bowel nodule (BN) MRI images and TVUS clips.

[0104] In essence, in some examples, MRI volumes labelled with both POD obliteration and bowel nodules may be paired with two separate TVUS videos: one based on the POD label and another on the bowel nodule label. Since the TVUS dataset may not include any samples with concurrent positive annotations for both POD obliteration and BN, a multi-label mixup strategy may be used. This technique integrates the two TVUS scans to create ( ^^^^^^, ^^^^^^) which serves as the final input for training the TVUS encoder as shown below:(Equation 2B)

[0105] , and ^^ is apredefined

[0106] Although TVUS and MRI may share identical labels, the present techniques permit the labels to be mixed empirically. Consequently, beneficially, this strategy enables the TVUS stream of the model to manage the classification of multiple signs using a single encoder, rather than creating separate encoders for each sign. Advantageously, maintaining a shared encoder helps the TVUS encoder learn more generalized feature representations while also significantly reducing the overall complexity of the model.

[0107] Once training is finalized, the model can perform classification using any combination of the available data modalities.

[0108] The method 300 may comprise S306 using the Learnable Cross-modal Knowledge Distillation (LCKD) model. The use of LCKD model enhances the performance of the entire AI model 300 depicted in Figure 3. However, the skilled person will appreciate that the use of the LCKD model is not necessary for the AI model 300 to work. The skilled person will appreciate that the LCKD model / method 500 may also be used on its own, without the other steps performed in method 300, to determine POD obliteration or any other sign of endometriosis and thus determine the probability of the diagnosis of endometriosis.

[0109] The creation of the LCKD model is further described in Figures 4 and 5.

[0110] The method 300 may comprise S310 extracting, by an MRI feature extractor, at least one MRI embedding from the endometriosis MRI dataset in an MRI stream.

[0111] This step comprises initializing the MRI feature extractor in the MRI stream. The MRI stream uses the MRI encoder from the pre-trained MRI feature extractor, with ^^^^= ^^θ^^^^( ^^^^).

[0112] The method 300 may comprise S312 extracting, by a TVUS feature extractor, atTVUS embedding from the TVUS dataset in a TVUS stream.

[0113] This step comprises initializing the TVUS feature extractor embedding extractor from the pre-trained TVUS dataset in the TVUS stream. The TVUS stream uses the embedding extractor from the pre-trained TVUS model, with ^^^^= ^^θ^^^^( ^^^^). For instance, R (2+1)D may be used to extract useful features from each TVUS video as a vector. More calculations can be subsequently applied to the extracted vector.

[0114] The skilled person will appreciate that the multimodal analysis of the endometriosis MRI and TVUS data sets are designed to produce an accurate classification between the two modalities, even when one modality is missing from the inputs to the AI model. Such missing modality analysis capability is extremely beneficial since endometriosis patients normally have results from just one of the two examinations (either MRI or TVUS) available for testing. This also enables the training of the method of the present techniques to use unpaired TVUS and MRI datasets, which are matched based only on their sign of endometriosis label (e.g. POD obliteration label and / or any other sign of endometriosis label)

[0115] The method 300 may further comprise S314 measuring uncertainty in the MRI stream.

[0116] The method 300 may further comprise S316 measuring uncertainty in the TVUS stream.

[0117] Measuring uncertainty in each one of the streams may be performed e.g. by a random network prediction module. Accordingly, the method may comprise initializing at least two random network prediction modules to determine the uncertainty in the TVUS stream and the MRI stream. Thus, both feature extractors from the MRI and TVUS streams are followed by Random Network Prediction (RNP) modules, referred to as ^^ ^^ ^^^^and ^^ ^^ ^^^^respectively, which measure the uncertainties of each stream. A stream refers to a flow of data that is analyzed and processed in real-time, or nearly-real time according to the techniques disclosed herein. For example, a TVUS stream corresponds to a flow of data processing TVUS results, and an MRI stream corresponds to a flow of data processing MRI results.

[0118] The ^^ ^^ ^^^^has a fixed-weight random network denoted by ^^^^ ^^( ^^^^) (similarly, ^^ ^^ ^^^^with ^^^^ ^^( ^^^^)) and a learnable prediction network ^^^^ ^^( ^^^^) (similarly for ^^ ^^ ^^^^^^ ^^ ^^ℎ ^^^^ ^^( ^^^^)).

[0119] Beneficially, the learnable prediction network may haveoutput space but a different structure from the fixed-weight randomwhere the capacity of the learnable prediction network is smaller than the fixed-weight random network to prevent potential trivial solutions. More specifically, the fixed-weight random network may be a neural network with three 1-dimensional convolutional layers. LeakyReLU activation functions may be applied after the first and second convolutional layers. The learnable prediction network may be a neural network with two 1-dimensional convolutional layers. A LeakyReLU activation functions may be applied after the first convolutional layer.

[0120] The method of the present techniques may also comprise training the RNP modules based on the minimisation of a loss function computed from the difference between outputs (in the same stream) by the prediction and random networks. The prediction network fits better the random network outputs of samples (i.e., with low uncertainty), populating denser regions of the feature space; but the fitting will be worse (i.e., with high uncertainty) for samples belonging to sparser regions. The minimization of Mean Squared Error (MSE) between the outputs of the prediction and random networks enables the prediction network to learn the uncertainty measuring function.

[0121] The training of the RNPs is based on the minimisation of a loss function computed from the difference between outputs by the prediction and random networks, as follows: (Equation 3)that the embedding space is in a two-dimensional orthogonal coordinate system, the samples that are easy to be classified should be closely gathered at the far end of the first(1,1) three(- 1,-1) quadrants, and the samples that are not easy to be classified should be sparsely scattered at (0,0). Consequently, embeddings in dense regions have low predictive uncertainty from the mean squared error (MSE) between ^^^^ ^^( ^^^^) and ^^^^ ^^( ^^^^) for MRI, and between ^^^^ ^^( ^^^^) and ^^^^ ^^( ^^^^) for TVUS. On the other hand, embeddings in sparsely populated regions (e.g., embeddings from blank input data representing missing modality) have high predictive uncertainty. Uncertainties in each one of the streams may be represented as a scalar value.

[0123] The method 300 further comprises S318 cross-modally fusing the MRI and TVUS embeddings. This step comprises cross- the MRI and TVUS embeddings by multiplying the at least one MRIuncertainty in the TVUS stream and by multiplying the at least one TVUS embedding by the measured uncertainty in the MRI stream.

[0124] In the multimodal fusion stage, the present method deals with a multimodal classification task that relies on diverse data modalities (MRI and TVUS) with larger domain shifts than different MRI modalities. Additionally, the training data of the AI model is unpaired. Moreover, the testing of the AI model may be preferably performed with a missing modality. Accordingly, the method 300 of the present techniques may comprise producing a single scalar per modality to represent the modality uncertainty, i.e., ^^^^for the MRI stream and ^^^^for the TVUS stream, computed from the output of the RNP module to represent the modality uncertainty as follows: (Equation 4)

[0125] The method of the present techniques comprises multiplying the MRI embedding ^^^^^^with the cross-modal scalar uncertainties ^^^^and multiplying TVUS embedding ^^^^^^with the cross-TVUS embedding ^^^^^^are multiplied by the scalar uncertainties ^^^^and ^^^^, respectively (see Fig.2(C)). Such cross-modal fused features with uncertainties for the MRI and TVUS modalities can be denoted by:(Equation 5)

[0126] The TVUS embeddings may be 1-d vectors with a length of 512. The MRI embeddings may be 1-d vectors with a length of 768. Each sample may have its own embedding and it will be 1 MRI embedding for 1 TVUS embedding. In the linear projection layer, the final length of the vector may be 512.

[0127] For example, during the multimodal training stage, the method 300 may use the “5- fold cross validation” strategy (the TVUS classifier pre-training may use the same folds) on the endometriosis MRI dataset. With each fold, the multimodal fusion model may be trained for 100 epochs, a 0.05 weight decay, a batch size of 7 and a learning rate of 1e-4. The RNP modules may be optimised with Stochastic Gradient Descent (SGD) and 0.9 momentum. In this multimodal training, it is possible to keep the TVUS classifier frozen (i.e., not fine-tuned during multimodal training) or unfrozen (i.e., fine-tuned during multimodal training). In some examples, RNP modules may be initialised with random parameters. An optimizer may be used during training to improve the performance of the model by updating the model's parameters. A SGD optimizer adjusts the parameters based on the calculated gradients of the loss function, allowing the model to gradually converge towards an optimal solution. By iteratively updating the parameters using optimization algorithms such as gradient descent, the optimizer helps the model find the optimal set of parameters that minimize the loss and improve the model's accuracy and predictive capabilities. Beneficially, using a frozen TVUS feature extractor can ensure good classification performance of the model for TVUS and also saves training resources.

[0128] The method of the present technique 300 further comprises S320 concatenating the cross-modal fused MRI and TVUS embeddings.

[0129] The method 300 may further comprise S322 passing the concatenated MRI and TVUS embeddings to at least one linear layer. This produces a classification prediction of at least one sign of endometriosis (e.g. POD obliteration) which is indicative of endometriosis diagnosis.

[0130] The cross-modal fused features ^^̃^^^^and ^^̃^^^^are concatenated and passed to a final linear layer, denoted by ^^⋋: ^^ × ^^ ^ ∆ which produces the classification prediction.

[0131] The whole model may be optimised as follows:(Equation 6)

[0132] During training, the present method may iteratively optimise ^^^∗^, ^^^∗^from equations 3 and strategy for the Entropy so that

[0133] The AI model is preferably tested only with one modality (either MRI or TVUS input). During testing with only MRI modality, the input ^^^^= 0^^^^× ^^^^× ^^^^ is provided to the TVUS stream. Similarly, during testing with only the TVUS modality, the MRI input is set with ^^^^= 0^^× ^^× ^^. In a case where TVUS is missing, ^^^^will be high because the TVUS modality is uncertain, resulting in an amplification of the MRI embedding with equation 5 (conversely for missing MRI).

[0134] In some examples, where more than one sign of endometriosis is used in training at the same time (such as POD obliteration and bowel nodules) at least one linear layer may comprise a multi-head attention (MHA) layer.

[0135] Considering the inherent connections between various endometriosis signs and their differing occurrence rates, a multi-label classification module that harnesses these relationships and effectively manages the challenges of imbalanced learning has been developed. To capture the inter-label relationships, a Multi-Head Attention (MHA) layer may be employed.

[0136] In some examples, to implement the MHA layer, the cross-modal fused features ^^̃ may be processed through two fully connected (FC) layers, producing initial classification outputs for each label. These outputs may be stacked and fed through the MHA layer, which identifies correlations among the labels. The results from the MHA may be unstacked and passed through another pair of classification layers, generating the multi-label probabilities needed for diagnosing endometriosis. The multi-label classification module comprising the MHA layer may be trained using the Two-Way Loss (TWL) function, which has been demonstrated to be effective in addressing class imbalances in multi-label learning environments. The TWL function specifically addresses class imbalance in multi-label learningby distinguishing between different classes and samples, offering a more effective solution to the imbalance problem compared to the traditional binary cross-entropy loss function.

[0137] In some examples, the TWL function may be minimized by: (Equation 6B)

[0138] The TWL function may be optimized across n samples and C labels using:(Equation 6C) pre- ^^^^ _. bowel nodules, ovarian endometrioma and uterosacral ligament nodules can be detected. This result may be outputted to a user of the AI model as an indication of the sign of endometriosis (e.g. POD obliteration and / or bowel nodules and / or ovarian endometrioma and / or uterosacral ligament nodules) and / or the likelihood of endometriosis diagnosis.LCKD model

[0142] The AI model 300 of the present techniques has the capability to diagnose endometriosis using medical imaging techniques, specifically magnetic resonance imaging (MRI) and transvaginal ultrasound (TVUS) modalities. To further enhance the AI model's performance and address the challenge of missing modalities during diagnosis, Learnable Cross-Modal Knowledge Distillation (LCKD) method / model 500 may be implemented within the creation of AI model 300 as described in step 308 of Figure 3.

[0143] Figure 4 shows diagrammatically a framework for creating the LCKD model 500. A method for creating the LCKD model 500 is shown in Figure 5. The LCKD model 500 may be used in the AI model 300 to enhance the accuracy of method 300. Thus, the skilled person will appreciate that the LCKD model 500 may be combined with the AI model 300. The skilled person will appreciate that the LCKD model / method 500 may also be used on its own, without the other steps performed in method 300, to determine at least one sign of endometriosis (e.g. POD obliteration and / or bowel nodules and / or ovarian endometrioma and / or uterosacral ligament nodules) and thus determine the probability of diagnosis of endometriosis.

[0144] Learnable Cross-modal Knowledge Distillation (LCKD) model 500 is able to handle missing modalities in both training and testing by automatically identifying important modalities and distilling knowledge from them to learn the parameters that are beneficial for all tasks while training for other modalities. In other words, LCKD is a method / model that leverages the knowledge learned from one modality to assist in the analysis of another modality and automatically detects the “teacher” and “student” modalities. In the case of the present AI model 300, LCKD may be used to bridge the gap when one of the modalities (endometriosis MRI or TVUS) is missing for a particular patient more efficiently. Beneficially, by distilling cross-modal knowledge, the LCKD model can utilize the available modality's features to fill in the missing information more precisely and further improve diagnostic accuracy.

[0145] In the present AI model, there are only two modalities: endometriosis MRI and TVUS. Therefore, the LCKD method 500 is configured to accommodate this specific setting. The skilled person will appreciate that the present LCKD model 500 can be adjusted to process more than two modalities.

[0146] In the LCKD model 500, the teacher network is trained on the available modality (MRI or TVUS or both) to provide guidance and knowledge to the student network, which analyzes the other modality. Beneficially, this setup allows the student network to learn from the teacher network's knowledge, enabling the utilization of cross-modal information.

[0147] The knowledge distillation (KD) process involves transferring the knowledge from the teacher network to the student network. The student network receives guidance from the teacher network during training and learns to mimic its behaviour. Thus, beneficially, this knowledge transfer facilitates the student network in filling in missing modalities' features using the available modality.

[0148] Creating the LCKD model 500 may comprise S502 pre-processing datasets. The endometriosis MRI and TVUS images may be pre-processed to ensure their compatibility. This step may involve normalization and / or resizing and / or data augmentation. For example, the following techniques may be used to pre-process the endometriosis MRI and TVUS datasets: mirror operation technique and / or Gaussian noise technique and / or brightness and contrasting enhancement.

[0149] Creating the LCKD model 500 may comprise pre-training. This step corresponds to the pre-training steps S302, S304 and S306 of Figure 3. Thus, the description of these steps is omitted in the context of creating the LCKD model. The skilled person will appreciate that when the LCKD model is combined with the AI model, pre-training is done only once, and does not have to be repeated. The LCKD model may be used on its own. Thus, beneficially, the LCKD model does not require pre-training, but it can have at least one pre-trained encoder input as initialisation. The encoders may be pre-trained as depicted in Figure 3.

[0150] Creating the LCKD model 500 comprises S504 electing a teacher. The teacher election procedure will be performed on the available modality using the endometriosis MRI or TVUS dataset to automatically detect the most suitable modality as the “teacher”. Specifically, electing a teacher comprises using a validation process. The validation process examines each single modality and determines how well it will perform on the validation set. Then, the highest performing modality is picked as the teacher.

[0151] The validation process is applied for each task k (for ^^ ^^ {1, … , ^^}), the modality with the best performance is selected as the teacher ^^( ^^). Formally, we have: (Equation 7)

[0152] Where I indexes different modalities, ^^ (. ; Θ) is the LCKD segmentation model parametrised by Θ including the encoder and decoder parameters { ^^^^ ^^ ^^, ^^^^ ^^ ^^} ^^ Θ, ^^ ^^ ^^ ^^(. , . ) is the function to calculate Dice score. ^^

[0153] This procedure may occur before processing all modalities { ^^( ^^)} ^^=1 ∈ Μ, where ^^( ^^) ^^^^ Χ denotes the ^^^^ℎdata sample and ( ^^) indexes the modality. The skilled person will appreciate that the LCKD framework in Figure 4 shows more than two modalities. However, in the present implementation, only two modalitiesor MRI) are present.

[0154] a promising performance has either MRI image or (either TVUS or MRI)modalities, ^^(2)is selected as a teacher, { ^^1, ^^3, … , ^^^^} are the students, and ^^^^( ^^ ^^ ^^ℎ ^^ ≠ 2) is assumed to be absent. ^ 155] Subsequently, the modalities are encoded to output features { ^^( )^ [0^^} ^^=1 individually. For the available modalities, namely ^^1, ^^( ^^−1), ^^( ^^+1)… , ^^^^,out between each pair of teacher and student modalities. However, for the absent modality ^^^^, its features ^^^^are produced through a missing modality feature generation process from the available features ^^1, ^^( ^^−1), ^^( ^^+1)… , ^^^^.

[0156] The teacher is a modality with a promising performance. In the determination of the endometriosis method, the teacher will be the available modality. The teacher election procedure is further introduced to automatically elect a proper teacher for different tasks.

[0157] In some examples or as an alternative, S504 electing a teacher may comprise electing worst-student oriented teacher.

[0158] In this step, the encoder associated with the modality that shows theperformance for a particularas a student, learning from the teacher, which is the encoder of the modality that performs better for that label. Beneficially, this approach allows the modalities to learn from each other, enhancing their strengths and compensating for their weaknesses.

[0159] The teacher modality may be elected by comparing the performance of the two modalities at each training iteration. For instance, during the first training epoch, the teacher modality may be chosen randomly. After the first epoch is complete, the model’s performancemay be assessed using a validation dataset. The validation dataset may be created through stratified randomdataset. This evaluation assesses the model’s performance across different labels for each modality.

[0160] For example, if the signs of endometriosis are POD obliteration and bowel nodules, this process may generate the area under the receiver-operating curve classification results on the validation set for both POD obliteration ( ^^ ^^ ^^ ^^^^^^^^ ^_^^^ ^^ ^^) and bowel nodules ( ^^ ^^ ^^^^^^^^ ^_^^^ ^^) for MRI data, and POD obliteration ( ^^ ^^ ^^ ^^^^^^^^_^^^^ ^^ ^^) and bowel nodules ( ^^ ^^ ^^^^^^^^_^^^^ ^^) for TVUS data. By comparing these four metrics, the worst-performing student and its corresponding modality may be identified. Then, the other modality may be designated as a teacher ^^( ^^)for the subsequent training epoch. Specifically, in the ^^^^ℎepoch, the worst performing modality may be defined as: (Equation 7B)

[0161] The modality demonstrating the worst performance in the ^^^^ℎepoch may be determined by: (Equation 7C)

[0162] As a result, the teacher and student modalities for the ( ^^ + 1)^^ℎepoch are:(Equation 7D)

[0163] During the training process, the teacher modality may not be static. The teacher modality may change dynamically due to the differing learning complexities inherent in eachmodality and label. Beneficially, this approach allows the two modalities to take turns acting as the teacher, thereby promoting effective mutual knowledge distillation.

[0164] Creating the modal knowledge.

[0165] As shown into the encoder parametrised by ^^^^ ^^ ^^,^^( ^^)= ^^^^ ^^ ^^( ^^( ^^)) (Equation 8)

[0166] The cross-modal knowledge distillation (CKD) is defined by a loss function that approximates all modalities features to the teacher modalities in a pairwise manner for all tasks as follows:^^∈ ^^; ^^, ^^≠ ^^ (Equation 9)

[0167] where ‖ ^^^^− ^^^^‖ ^^ presents the p=norm operation and modality ^^ is assumed to be missing. The minimisation of this loss pushes the model parameter values to a point in the parameter space that can maximize the performance of all tasks for all modalities.

[0168] Because of the knowledge distillation between each pair of teachers and students, the features of modalities in the feature space ought to be close to the “genuine” feature that can uniformly perform well for different tasks. Still assuming that modality ^^ is missing, the missing feature ^^( ^^)can thus be1 ( ^^−1) ( ^^+1) ^^features ^^ , ^^ , ^^ … , ^^ : ^^ ^( ^1 ^^)= ∑ ^^^^^^ − 1 ^^=1; ^^≠ ^^ (Equation 10)

[0169] For KD, we use a certain metric (L1 or L2 distance) to drag closer the features from the two modalities. That is the missing one with the available one, because their

[0170] Creating the LCKD model 500 comprises S508 training the teacher-student pair. The teacher-student pair will be trained using the available modality (endometriosis MRI or TVUS or both) and guided by the teacher network's knowledge. During training, the student network aims to mimic the behaviour and predictions of the teacher network. Beneficially, this process enables the student network to leverage cross-modal knowledge and fill in missing modality features. The training process may be conducted iteratively. Similarly, the teacher election procedure may be conducted iteratively.

[0171] All features encoded from Equation 8, or generated from equation 10, are then concatenated to be fed into the decoder parametrised by ^^^^ ^^ ^^for(Equation 11)

[0172] Where ^^̃ ∈ ^^ is the prediction of the task.

[0173] The training of the LCKD model is achieved by minimising the following objective function: ^^^^ ^^ ^^( ^^; Θ) = ^^^^ ^^ ^^ ^^( ^^, ^^^^ ^^ ^^, ^^^^ ^^ ^^) + ^^ ^^^^ ^^ ^^( ^^, ^^^^ ^^ ^^) (Equation 12)

[0174] where ^^^^ ^^ ^^ ^^( ^^, ^^^^ ^^ ^^, ^^^^ ^^ ^^) is the objective function for the whole task (e.g. cross- Entropy and Dice losses are adopted for a sign of endometriosis (e.g. POD obliteration detection)); α is the trade-off factor between the task objective and cross-modal objective. If α= 0, the performance of the model drops, but when α>0, results improve. A suitable value for α may be 0.1.

[0175] Testing of the LCKD model may be based on taking all image modalities available in the input to produce the features from equation 8, and generating the features from the missing modalities with equation 10 which are then provided to the decoder to predict the segmentation with equation 11.

[0176] At the inference stage, as shown for instance in Figure 4 or in Figure 6 when the LCKD model is used in the AI model 300 or on its own, when one modality is missing for aspecific patient (MRI or TVUS), the trained student network can utilize the available modality to generate predictions by incorporating the cross-modal knowledge learned during training. This ensures that the can still even if one is unavailable. Thus, by leveraging cross-modality features, ensuring reliable diagnoses even when one modality is unavailable. Using the AI to determine endometriosis

[0177] Figure 6 shows a computer-implemented method 400 of using the AI model and / or the LCKD model to detect at least one sign of endometriosis (POD obliteration and / or bowel nodules and / or ovarian endometrioma and / or uterosacral ligament nodules) and thus determine endometriosis.

[0178] Method 400 comprises S408 obtaining an MRI result or TVUS result of a subject. The skilled person will also appreciate that the method works when S408 comprises obtaining an MRI result and TVUS result of a subject. MRI result is an MRI scan image of a particular patient. TVUS result is a video of the TVUS examination of a particular patient. The method 400 further comprises S410 inputting the MRI result and / or TVUS result to the AI model and / or the LCKD model. The AI model is described in Figure 3. The LCKD model is described in Figure 5.

[0179] The subject may be a patient. The MRI result or TVUS result may be inputted into the method / AI model / LCKD model by a user. The skilled person will appreciate that the subject may input the MRI result and / or TVUS result of a subject to the AI model and / or LCKD model. The user may be a medical professional. Alternatively, the patient may be able to input the MRI result and / or TVUS result into the method / AI model and / or LCKD model. The MRI result may be a T2 SPACE image. The TVUS result may be a video of a sliding sign examination. The skilled person will appreciate that any other similar formats may be used, but the MRI result is preferably in the same format as the endometriosis MRI dataset and the TVUS result is preferably in the same format as the TVUS dataset utilised to build the AI model and / or the LCKD model.

[0180] The MRI result and / or TVUS result of a subject may be obtained by a direct upload from a user to a computing device which is configured to implement the method of the present techniques / AI model and / or the LCKD model. The MRI result and / or TVUS result of a subject may be obtained, from a user, via a communication network (e.g. internet, Bluetooth etc.).

[0181] The skilled person will appreciate that the method 400 may comprise obtaining an MRI result and / or TVUS result of a subject and the method 400 may further comprise S410 inputting the MRI result and / or TVUS result into the AI model and / or the LCKD model.

[0182] The method 400 further comprises S424 detecting at least one sign of endometriosis (e.g. POD obliteration and / or bowel nodules and / or ovarian endometrioma and / or uterosacral ligament nodules). The at least one sign of endometriosis can be classified / detected by the AI model and / or the LCKD model.

[0183] The method 400 further comprises S426 outputting the result of the sign of endometriosis detection. In an instance where the sign of endometriosis is POD obliteration, the POD obliteration is a strong indicator for the diagnosis of endometriosis. Thus, the method of the present techniques 400 may output, to the user, the conclusion on whether the sign of endometriosis (e.g. POD obliteration) has been detected. Alternatively, or additionally the method of the present techniques 400 may output a score indicating the probability of the sign of endometriosis (e.g. POD obliteration). This score may be shown in percentage certainty. The classification may be yes (e.g. POD obliteration detected) or no (e.g. no POD obliteration detected). These results may be displayed on a display of a device that is configured to implement the method of the present techniques and / or the AI model and / or the LCKD model. Alternatively, the results may be sent to a user’s device of choice via a secure communication technique (e.g. private / password-protected email address etc.)

[0184] Interaction between the AI model and the LCKD model

[0185] The skilled person will appreciate that the AI model may be implemented on its own, without the LCKD model. In other words, the method of Figure 3 may be implemented without the S308: Using LCKD model. A sign of endometriosis (e.g. POD obliteration and / or bowel nodules and / or ovarian endometrioma and / or uterosacral ligament nodules) may be detected by using only the AI model of Figure 3.

[0186] The skilled person will appreciate that the LCKD model may be implemented on its own, without the AI model. In other words, the method of Figure 5 may be used on its own. At least one sign of endometriosis (e.g. POD obliteration and / or bowel nodules and / or ovarian endometrioma and / or uterosacral ligament nodules) may be detected by using only the LCKD model of Figure 5.

[0187] The skilled person will also appreciate that the LCKD model and the AI model may be combined together as depicted in Figure 3. This means that the steps in Figure 3 (relating to establishing the AI model) may be combined with the steps of creating the LCKD model(relating to establishing the LCKD model) in Figure 5. Beneficially, this combination yields an improved result in the detection of at least one sign of endometriosis.

[0188] In this implementation, the LCKD model and the AI model may be combined together in a stacking way. For instance, the AI model stacks onto the LCKD model as shown in Figure 3. More specifically, there are two modalities MRI images and TVUS videos. The two modalities are passed into the LCKD model to deal with the following two situations: 1) if two modalities are all available, then the Knowledge Distillation process of the LCKD model will be performed; 2) if one of the modalities is missing, then the missing features will be predicted by the LCKD model. Then, the features for two modalities (either available or predicted) will be available for the AI model uncertainty estimation, and classification prediction in the end.

[0189] Example 1:wherein a sign of endometriosis is POD obliteration

[0190] In a case study, the dataset consists of a female pelvic MRI dataset for self- supervised pretraining, an MRI endometriosis dataset, and a TVUS endometriosis dataset. The female pelvic MRI dataset contains 8,984 T2 MRI volumes from females aged 18-45 with endometriosis-like symptoms. These scans were performed at different sites with different protocols, most likely containing signs of other medical conditions and the examination area is not necessarily aligned with the diagnostic area of endometriosis. The MRI endometriosis dataset consists of 88 T2 SPACE MRI pelvic scans of females aged between 18-45, conducted in multiple clinics under a standard protocol, and including 19 POD obliteration cases. These endometriosis MRIs are focused on the diagnostic region of POD obliteration near the uterus. The TVUS endometriosis dataset has 749 “sliding sign” video clips, including 103 negative sliding sign (POD obliterated) cases.

[0191] In this case study, we pre-train the 3D MAE network for 195 epochs from scratch on the female pelvic MRI dataset, for instance, using AdamW / Adam optimiser, a batch size of 3 and a learning rate of 1e-3. In the multimodal fusion stage, we applied the encoder followed by a linear layer, to adjust the output size (768) to be the same as the output size (512) of the TVUS encoder, as the MRI feature extractor. For the ResNet(2+1)D network, we pre-trained on the Kinetics-400 dataset and trained for 30 epochs on the TVUS endometriosis dataset, using Adam / AdamW optimiser, a learning rate of 1e-5 and a batch size of 30. For this training of the TVUS classifier, we conducted a “5-fold cross validation” strategy to split the dataset into 5 folds by stratified random sampling, using 4 folds for training and 1 for testing. During the multimodal training stage, we performed the “5-fold cross validation” strategy (the TVUS classifier pre-training used the same folds) on the endometriosis MRI dataset. With each fold, the multimodal fusion model was trained for 100 epochs, using AdamW

[0015] , a 0.05 weightdecay, a batch size of 7 and a learning rate of 1e-4. The RNP modules are optimised with SGD and 0.9 momentum. In this multimodal training, we either keep the TVUS classifier frozen (i.e., not fine-tuned during multimodal training) or unfrozen (i.e., fine-tuned during multimodal training).

[0192] In this case, we also evaluated with the Area Under the Receiver Operating Characteristic Curve (AUC) for the five folds, for which we show the mean and standard deviation results. Python 3.9.7 with PyTorch 1.7.1 was used for all experiments. The 3D MAE pre-training was run on 3 NVIDIA RTX 3090 GPU with 24 GB VRAM in 98 hours. The multimodal model was trained on 1 NVIDIA RTX A6000 GPU, taking 4 hours per fold and requiring 47 GB GPU memory.

[0193] The results of the case study are shown in Figure 7. The POD obliteration classification AUC results are in Figure 7. The ResNet(2+1)D model (single modality TVUS training and testing) achieves AUC= 92.38% in POD obliteration classification, which is slightly below the AUC= 96.5% because of the 5-fold cross validation experiment.3D ViT trained from scratch (single modality MRI training and testing) using the MRI endometriosis dataset achieves a low AUC= 65.03% because of the small dataset size. Self-supervised pre-training (PT) on the large-scale female pelvic MRI dataset based on 3D MAE substantially improves the ViT results to AUC= 90% (single modality MRI training and testing). The Multi-modal Fusion (MMF) without pre-training has a low AUC=63.48% (multimodal training, MRI testing with missing TVUS). Based on MAE pre-training, our multimodal learning model with an unfrozen (i.e., learnable) pre-trained TVUS classifier reaches AUC= 91.79% (MRI testing with missing TVUS). Freezing the TVUS classifier for multimodal learning further increases the POD detection AUC to 92.14% (MRI testing with missing TVUS). For TVUS testing with missing MRI, the POD detection improves slightly from single modality TVUS training and testing, with an AUC of 92.57%.

[0194] Example 2: where signs of endometriosis include POD obliteration and bowel nodule

[0195] In another case study, the dataset consists of a female pelvic MRI endometriosis dataset and a TVUS endometriosis dataset. The MRI endometriosis dataset comprised of 171 T2 SPACE MRI pelvic scans from women aged 18-45, including 63 scans showing varying levels of POD obliteration and 11 scans containing bowel nodules. Uterine segmentation was carried out through manual annotation using ITK-SNAP. All 3D volumes were standardized to the Right-Anterior-Superior (RAS) coordinate system to ensure consistent anatomical alignment, then resampled to a uniform spacing of 1×1×1mmusing SimpleITK, and ultimatelyresized to 64 × 128 × 128. The TVUS sliding sign dataset comprised of 749 video clips, each approximately 10 seconds long, featuring the ’sliding sign,’ including 103 cases where the sliding sign was negative (indicating POD obliteration). Additionally, the TVUS bowel nodule dataset contains 519 video clips of bowel TVUS scans, each about 5 seconds in duration, with 35 cases presenting bowel nodules. During preprocessing, the top 70 pixels of each frame were automatically cropped to focus on the fan area, followed by uniform temporal sampling of 32 frames per sequence, with spatial resolution adjusted to 112 × 112 using bilinear interpolation.

[0196] During the pre-training phase, the 3D MAE network utilizes an encoder composed of 12 transformer blocks and a 768-dimensional linear projection layer. The masking ratio is set to ^^ = 0.7. The decoder is made up of 8 transformer blocks. The entire network is trained from scratch on the female pelvic MRI dataset over 195 epochs. Model parameters are fine-tuned using the AdamW optimizer (Loshchilov and Hutter, 2017), with ^^1 set to 0.9 and ^^2 to 0.95. The batch size is fixed at 3. The initial base learning rate is set at 1e-3 and is gradually reduced following a linear scaling rule: ^^ ^^ = ^^ ^^ ^^ ^^_ ^^ ^^ × ^^ ^^ ^^ ^^ℎ ^^ ^^ ^^ ^^∕256 as described in (He et al., 2022). For all subsequent training stages, a 5-fold cross-validation strategy was used to partition the endometriosis MRI dataset and both TVUS datasets into 5 folds through stratified random sampling, with 4 folds dedicated to training and 1 fold reserved for testing. This initial split was consistently applied throughout all experiments. Furthermore, it was ensured that the same patients were not included in both the training and validation / testing sets.

[0197] During the segmentation training phase, in each fold, the 3D MAE pre-trained encoder and decoder undergo training for 200 epochs, utilizing the AdamW optimizer with a learning rate set at 1e-4 and a batch size of 3. The segmentation and classification heads have dimensions of 1536 and 768, respectively. Adaptive histogram equalization and random cropping are applied as data augmentation techniques in the training process.

[0198] In the multi-modal multi-label classification phase, ^^ = 4 for TVUS video mixup. The MRI feature extractor is the encoder pre-trained by 3D MAE, with an extra linear layer to change the output dimension from 768 to 512 to fit the TVUS encoder output. The TVUS feature extractor is ResNet(2+1)D model, which has 18 R(2+1) convolutional layers. Batch normalization is used following each convolution and preceding the Rectified Linear Unit (ReLU) activation. The network incorporates an Adaptive Average Pooling layer and has been pre-trained on the Kinetics-400 dataset. For each fold, the model is trained over 200 epochs using the AdamW optimizer, with a weight decay set at 0.05. The batch size is configured to 16, with gradient accumulation across 4 iterations, effectively increasing the batch size to 64. The initial learning rate is set to 1e-3. To simulate scenarios where a modality might be missingwithout requiring specific training modifications, one modality is randomly omitted in 50% of the iterations. The first set of FC layers is configured with a dimension of 64, while the MHA has 4 heads.

[0199] For MRI images, data augmentation techniques include random cropping, mirror flipping, contrast enhancement, simulated low resolution, Gaussian blur, Gaussian noise, brightness adjustment, gamma correction, and multiplicative brightness adjustment. No data augmentation techniques were applied to the TVUS dataset.

[0200] In the five-fold cross-validation framework, the AUCs were calculated for four tasks on the respective validation set for each fold: AUC for classifying POD obliteration using MRI data (MRI_POD), AUC for classifying bowel nodules using MRI data (MRI_BN), AUC for classifying POD obliteration using TVUS data (TVUS_POD), and AUC for classifying bowel nodules using TVUS data (TVUS_BN). For each task, the AUC values were gathered from the five folds at every epoch, then the mean and standard deviation were computed. This procedure was repeated for each epoch, generating a set of mean and standard deviation values for each task at each epoch. Next, the AUC values were calculated for all four tasks at every epoch, followed by computing the mean and standard deviation of these four AUCs. The highest mean AUC (AVG AUC) achieved across all epochs was identified as the model’s performance metric, also recording the AUC for each task at the epoch when this highest average occurred. For each epoch, the AUC values were determined for all four tasks and then both the mean and standard deviation of these AUCs was computed. The classification outcomes of are presented in Table 1. Table1

[0201] The results indicate that the presently disclosed Dynamic Mutual cross-modal Knowledge Distillation (DMKD) being a multi-modal version of the LCKD approach achieves the highest AUC across multiple diagnostic labels and, notably, achieves the best average AUC (AVG) of 0.8481.

[0202] Figure 8 is a schematic illustration of a computing device 800 which can be used to implement the methods of the present techniques. The methods of the present techniques as described in Figures 3, 5 and 6 are computer-implemented methods. These methods may be implemented on a computing device such the computing device 500.

[0203] Figure 8 comprises a computing device 800, which may be any suitable electronic device, e.g. a personal computer or computing device, a laptop, a tablet, a smart phone etc. It will be understood that this is a non-exhaustive and non- list of devices.at least one processor 502 coupled to memory 504. The at least one processor 502 may comprise one or more of: a microprocessor, a microcontroller, and an integrated circuit. The memory 504 may comprise volatile memory, such as random access memory (RAM), for use as temporary memory, and / or non-volatile memory such as Flash, read only memory (ROM), or electrically erasable programmable ROM (EEPROM), for storing data, programs, or instructions, for example. The computing device 500 typically comprises at least one user interface 506. The computing device 500 may comprise at least one display 508 for providing the results and / or data generated during the method described above. The device 500 may also comprise a machine learning module which trains, stores and executes the AI model of Figure 3 and / or the LCKD model of Figure 5. The device 500 may also be implemented in a cloud computing system. Accordingly, the method of the present techniques may be provided to a user as a web-based application, such as SaaS (Software as a Service). The AI model may be downloadable to a user's device.

[0204] Aspects of the present techniques have utility in the improved utilisation of imaging techniques such as TVUS and / or MRI in the diagnosis of endometriosis, namely by using a technical approach to supply a missing imaging modality. This may be beneficial for several reasons. For example, POD obliteration is one of the most easily detected signs of endometriosis on TVUS. However this sign is less easily detected by MRI by visualisation of strands of fibrotic tissue, indistinct tissue margins at the back of the uterus and lack of fluid filled spaces behind the uterus. The techniques disclosed herein can improve the detection of POD obliteration from MRI to almost the same level of detection as TVUS. Surgically POD obliteration can prevent laparoscopic visualisation of lesions behind the uterus (the most common site), will increase the duration of surgery and requires a higher surgical skillset to remove endometriosis and adhesions around the bowel and other organs. Eliciting this sign is especially useful for surgical triage to ensure a surgeon with an appropriate skillset undertakes the surgery, enough time is allocated and if the bowel is tethered a bowel preparation will be required. It also has implications for fertility as the fallopian tubes may be tethered or obstructed by the adhesions preventing functional transportation of eggs, sperm and embryos.

[0205] For bowel nodules, specialist sonography skills are required. Furthermore this sign is limited by the sonographic reach of the probe and nodules on the upper bowel can be missed and may be better detected by MRI. Improved detection of endometriosis bowel nodules canexplain symptoms of have bowel pain, constipation, diarrhea, nausea and anorexia. It can also reduce wasted time and resources spent exploring the gastrointestinal system using colonoscopies and endoscopies by gastroenterologists when a gynaecolgical referral is required. Misdiagnosis of irritable bowel syndrome can be avoided and targeted endometriosis therapies commenced in a timely way. More accurate detection also assists with surgical planning and triage.

[0206] Similarly, more accurate detection of signs of endometriosis such as ovarian endometriomas and / or uterosacral (USL) ligament nodules can influence treatment decisions about egg freezing and IVF for ovarian endometriomas, and decisions about medically supressing periods, undertaking surgery and monitoring kidney and / or bladder pain for nodules on the USL ligaments.

[0207] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification discussions utilizing terms such as "processing," "computing," "calculating," “determining”, analyzing” or the like, refer to the action and / or processes of a computer or computing system, or similar electronic computing device, that manipulate and / or transform data represented as physical, such as electronic, quantities into other data similarly represented as physical quantities.

[0208] In a similar manner, the term “controller” or "processor" may refer to any device or portion of a device that processes electronic data, e.g., from registers and / or memory to transform that electronic data into other electronic data that, e.g., may be stored in registers and / or memory. A “computer” or a “computing machine” or a "computing platform" may include one or more processors.

[0209] Reference throughout this specification to “one embodiment”, “some embodiments” or “an embodiment” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, appearances of the phrases “in one embodiment”, “in some embodiments” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more embodiments.

[0210] As used herein, unless otherwise specified the use of the ordinal adjectives "first", "second", "third", etc., to describe a common object, merely indicate that different instances of like objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.

[0211] In the claims below and the description herein, any one of the terms comprising, comprised of or which comprises is an open term that means including at least the elements / features that follow, but not excluding others. Thus, the term comprising, when used in the claims, should not be interpreted as being limitative to the means or elements or steps listed thereafter. For example, the scope of the expression a device comprising A and B should not be limited to devices consisting only of elements A and B. Any one of the terms including or which includes or that includes as used herein is also an open term that also means including at least the elements / features that follow the term, but not excluding others. Thus, including is synonymous with and means comprising.

[0212] It should be appreciated that in the above description of embodiments of the disclosure, various features of the disclosure are sometimes grouped together in a single embodiment, Fig., or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this disclosure.

[0213] Furthermore, while some embodiments described herein include some but not other features included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the disclosure, and form different embodiments, as would be understood by those skilled in the art. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0214] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the disclosure may be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

[0215] Similarly, it is to be noticed that the term coupled, when used in the claims, should not be interpreted as being limited to direct connections only. The terms "coupled" and "connected," along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. Thus, the scope of the expression a device A coupled to a device B should not be limited to devices or systems wherein an output of device A is directly connected to an input of device B. It means that there exists a path between anoutput of A and an input of B which may be a path including other devices or means. "Coupled" may mean that two or more elements are either in direct physical, electrical or optical contact, or that two or more elements are not in direct contact with each other but yet still co-operate or interact with each other.

[0216] Embodiments described herein are intended to cover any adaptations or variations of the present invention. Although the present invention has been described and explained in terms of particular embodiments, one skilled in the art will realize that additional embodiments can be readily envisioned that are within the scope of the present invention.

[0217] Where any or all of the terms "comprise", "comprises", "comprised" or "comprising" are used in this specification (including the claims) they are to be interpreted as specifying the presence of the stated features, integers, steps or components, but not precluding the presence of one or more other features, integers, steps or components.

[0218] Those skilled in the art will appreciate that while the foregoing has described what is considered to be the best mode and where appropriate other modes of performing present techniques, the present techniques should not be limited to the specific configurations and methods disclosed in this description of the preferred embodiment. Those skilled in the art will recognise that present techniques have a broad range of applications, and that the embodiments may take a wide range of modifications without departing from any inventive concept as defined in the appended claims.recognise that present techniques have a broad range of applications, and that the embodiments may take a wide range of modifications without departing from any inventive concept as defined in the appended claims.SUBSTITUTE SHEET (RULE 26)

Claims

The claims defining the invention are as follows:

1. A computer-implemented method for determining the likelihood of endometriosis, the method comprising: obtaining a magnetic resonance imaging (MRI) result or a transvaginal ultrasound (TVUS) result of a subject; inputting the MRI result or TVUS result into an artificial intelligence (AI) model and / or a Learnable Cross-Modal Knowledge Distillation (LCKD) model; and detecting, by the AI model and / or the LCKD model, at least one sign of endometriosis; and outputting the result of the at least one sign of endometriosis detection; wherein the at least one sign of endometriosis is at least one of: Pouch of Douglas (POD) obliteration, bowel nodules, ovarian endometrioma and uterosacral ligament (USL) nodules.

2. The method of claim 1, wherein the AI model comprises: extracting, by a magnetic resonance imaging (MRI) feature extractor, at least one MRI embedding from an endometriosis MRI dataset in an MRI stream; extracting, by a transvaginal ultrasound (TVUS) feature extractor, at least one TVUS embedding from a TVUS dataset in a TVUS stream; measuring uncertainty in the MRI stream; measuring uncertainty in the TVUS stream; cross-modally fusing the MRI and TVUS embeddings by multiplying the at least one MRI embedding by the measured uncertainty in the TVUS stream and multiplying the at least one TVUS embedding by the measured uncertainty in the MRI stream; concatenating the cross-modally fused MRI and TVUS embeddings; and detecting the at least one sign of endometriosis.

3. The method of claim 2, wherein the AI model further comprises: pre-training the MRI feature extractor based on an MRI dataset; and / or pre-training TVUS feature extractor based on an TVUS dataset.

4. The method of any one of claims 2 to 3, wherein the AI model comprises:matching MRI images with positive / negative sign of endometriosis labels of an endometriosis MRI dataset to TVUS videos of the TVUS dataset with a negative / positive sign of endometriosis.

5. The method of any one of claims 2 to 4, wherein the AI model comprises: passing the concatenated cross-modally fused MRI and TVUS embeddings to a linear layer.

6. The method of claim 3, wherein the endometriosis MRI dataset and the TVUS dataset are unpaired.

7. The method of claim 3 or 6, wherein the endometriosis MRI dataset comprises a plurality of MRI images, wherein at least one of the plurality of MRI images comprises at least one of: a labelled POD obliteration case, a labelled bowel nodule case, a labelled ovarian endometrioma case and a labelled uterosacral ligament nodule case.

8. The method of claim 3 and / or any one of claims 6 to 7, wherein the TVUS dataset comprises a plurality of sliding sign TVUS video clips, wherein at least one of the plurality of the TVUS video clips comprises at least one of: a labelled POD obliteration case, a labelled bowel nodule case, a labelled ovarian endometrioma case and a labelled uterosacral ligament nodule case.

9. The method of claim 3, wherein the MRI dataset is unlabelled and comprises a plurality of MRI images of a female pelvic region.

10. The method of any one of claims 2 to 9, wherein the AI model comprises: measuring channel-wise uncertainties in the MRI stream by a first random network prediction (RNP) module; and / or measuring channel-wise uncertainties in the TVUS stream by measuring uncertainty in the MRI stream by a second RNP module.

11. The method of claim 10, wherein both the first and the second RNP modules comprise a fixed-weight random network and a learnable prediction network.

12. The method of any one of claims 10 to 11, wherein the AI model comprises:training the first and the second RNP modules based on minimisation of a loss function computed from the difference between outputs by the prediction and random networks.

13. The method of any one of claims 10 to 12, wherein the AI model comprises: outputting, by the first RNP module, a single scalar per modality to represent the uncertainty; and outputting, by the second RNP module, a single scalar per modality to represent the uncertainty.

14. The method of any one of the preceding claims, wherein obtaining the MRI result or the TVUS result of the subject comprises receiving the MRI result and / or TVUS result by a medical professional or the subject.

15. The method of any one of the preceding claims, wherein the MRI result is an MRI image; and wherein the TVUS result is a sliding sign TVUS video clip.

16. The method of any one of claims 2 to 15, wherein the TVUS feature extractor is a convolutional neural network.

17. The method of any one of claims 2 to 16, wherein the MRI feature extractor is 3D vision transformer.

18. The method of any one of the preceding claims, wherein the detection of at least one of: POD obliteration, a bowel nodule, an ovarian endometrioma and an uterosacral ligament nodule is used for diagnosis of endometriosis.

19. The method of any one of the preceding claims, wherein the LCKD model comprises electing a teacher modality by a validation process.

20. The method of claim 19, wherein the LCKD model comprises distilling cross-modal knowledge between each pair of available teacher and student modalities by a loss function.

21. The method of claim 20, wherein the LCKD model comprises training the teacher and student modality pair, wherein during training the student modality mimics behaviour and predictions of the teacher modality.

22. A data processing device comprising means for carrying out the method of any one of the preceding claims.

23. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any one of claims 1 to 21.

24. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any one of claims 1 to 21.

Images