Method for medical imaging

Abstract

A method for analysing anatomical structure data such as, for example, arteriovenous fistula (AVF) data includes the following steps: Receiving 2D ultrasound images of the anatomical structure; genera

Description

FIELD The invention relates to a method. Particularly, but not exclusively, the method is a method for medical imaging. Particularly, but further not exclusively, the method may be for use in analysing structures such as, for example arteriovenous fistulas (AVFs), vascular structures and organs and anatomical landmarks. BACKGROUND The analysis of certain anatomical structures , for example, AVFs is a difficult task requiring specialised staff. A well-functioning AVF is essential for haemodialysis. Despite regular duplex ultrasound a significant number of AVFs fail. Technical limitations include human factors in stenosis identification is reduced. Haemodialysis is the most frequent renal replacement therapy. A well-functioning AVF is essential for most patients on haemodialysis while they await renal transplant, if appropriate, but the annual costs are high. Turbulent blood flow leading to high shear stress within the AVF is thought to lead to intimal damage causingflow limiting stenoses or aneurysmal changes. When undetected and untreated, these stenoses lead to AVF thrombosis, the patient’s lifeline. AVF thrombosis is the single most frequent cause of morbidity in patients on haemodialysis. For this reason, AVF surveillance is important, with angioplasty recommended in a dysfunctional AVF, when stenoses >50% are detected. However, DUS needs to be reported by skilled ultrasound operators who are able to build a mental picture of the AVF. Being operator dependent, DUS may be prone to error. Clinicians rely on these written reports and, in the absence of substantial trust between the clinicians and the vascular scientists, invariably further imaging will be required prior to treatment. Equally importantly, there is a national shortage of skilled ultrasound operators with DUS requiring approximately 20 minutes / patient to acquire the image and 10 minutes for each report. Additionally, up to 83% of the sonographers suffer repetitive strain injury (RSI), emphasising the importance of newtechnologiesthat require less skill, less time to acquire, and which can be interpreted directly by clinicians. Currently, AVFs are examined with the use of ultrasound, by trained vascular specialists, who identify areas of suspected stenosis by viewing the arteriovenous fistula in more than one angle, use doppler assessment of blood flow at these sites to identify locations of stenosis within an arteriovenous fistula. This process is skilled and hence, the accuracy of the identification of such stenoses, and reporting, is operator-dependent as it is with the analysis of many anatomical structures. For example, patent application “Method and Apparatus for Generating a Combined Three-Dimensional Ultrasound Image” filed on March 9,2021, discloses the use of inertial tracking systems, such as via inertial tracking systems to generate 3 D images. Aspects and embodiments were conceived with the foregoing in mind. SUMMARY Aspects relate to medical imaging and / or analysis of anatomical structures such as, for example, AVFs and generally relate to methods of imaging and analysing these structures. In accordance with a first aspect of the present disclosure, there is provided a method for analysing and / or imaging anatomical structures such as, for example,, AVFs. The method may process data and may comprising the following steps: Receiving 2D ultrasound images of an anatomical structure; Generating transformed data by performing a 2D transformation and an elastic transformation with the 2D ultrasound images; Creating synthetic images by processing the 2D ultrasound data and the transformed data with a diffusion model; Converting 2D ultrasound images, transformed data and synthetic images into 3D models of the anatomical structure using Cycle GAN; and Outputting 3D models of the anatomical structure . The overall process would be faster, more accurate, reliably triage, and allow for better surgical planning, and for future predictions of morbidity and mortality. There is also a technical benefit in data mining - by creating 3D models over time-the machine learning model could overtime predict failure of, for example, a stenosis and therefore allow prompt treatment prior to emergency stenotic events. The newsystem will allow stacks of ultrasounds to be taken from any angle reducing need for specialist capture, and automatically converted to 3D models. The 3D models will have automatic triaging for risk of events such as stenosis for example percentile values and locations before being flagged for senior review. The need for specialised staff for which there is a shortage is reduced. Technical limitations include human factors in stenosis identification is reduced. There is also a technical benefit in data mining - by creating 3D models over time - the machine learning model could, over time, predict failure of, for example, a stenosis and therefore allow prompt treatment prior to emergency stenotic events. Other benefits include augmented reality, AR and virtual reality, VR integration for surgical / educational benefit. Allowing surgeons to be able to map stenosis points and plan accordingly - also be able to use the AR map in surgery to accurately enter the stenosis at the right point. Data mining from annotation layers (2D CNN) may allow running wall and stenosis measurements to be taken (confirmed by clinicians) and therefore help in early prediction of failure rates and future stenoses via measurements of the stenosis and wall dimensions (changing over time). Triaging would be possible on a scoring system from these data points. Surgical review would be through the completed 3D model where a model can be created and viewed by surgeons for planning for future interventions, through AR overlay of 3D model on patients arm more accurate surgical interventions would be allowed, and in the future the potential for automating surgical interventions with existing machines (such as the DaVinci surgical robot) to allow for automated treatment of these flow-limiting stenoses, and aftercare. Currently nothing exists in the field that can do the same job. The data gleaned from the process can be mapped by systems to allow for accurate failure predictions. The models create a robust visual representation to be viewed by surgeons to assess for fistula formation. The system will also allow for a reduction in the need for skilled staff (expensive finite resource) and provide accurate triaging of results (by way of assessing automatically for stenosis and providing prediction for future failure) allowing for those patients with greater needs to be reviewed faster. The overall process would therefore be faster, more accurate, reliably triage, and allow for better surgical planning, and for future predictions of morbidity and mortality. 3 In some embodiments, the 2D ultrasound images are in a standardized DICOM format, optionally wherein the 2D ultrasound images are original DICOM US Series Images, DUSI. In some embodiments, the method further comprises storing the 2D ultrasound images. In some embodiments, the method further comprises using a 2D CNN to annotate the 2D ultrasound images to show exact vessel anatomical positions. Thus, being able to calculate vessel diameters and show loci of specific vessel point. In some embodiments, the method further comprises transforming the 2D ultrasound images with elastic and set transformations, optionally storing the transformed DICOM US Series Images, TDUSI. In some embodiments, the method further comprises annotating the 2D ultrasound images with a 2D Convolutional Neural Networks, CNN . In some embodiments, the method further comprises comparing TDUSI with original 2D ultrasound images, optionally assuring quality through independent operator (human) analysis. In some embodiments, the method further comprises passing DUSI and TDUSI into a diffusion model, where the computer learns to create similar images, and weighting with DUSI higher weights and TDUSI lower weights. In some embodiments, the method further comprises completing the diffusion model training and outputting Diffusion Model DICOM US Series Images, DMDUSI, optionally the 2D CNN is used to annotate the DMDUSI. In some embodiments, the method further comprises passing DUSI, TDUSI and DMDUSI for pass back training between Cycle GAN and 3D CNN, optionally weighting with highest, TDUSI middle and DMDUSI lowest, as a weighting strategy. In some embodiments, the method further comprises using the weighting strategy throughout training to dynamically alter weights of training. In some embodiments, the method further comprises completing the 3D Model training and outputting 3D models. In some embodiments, the method further comprises passingthe 3D Models to another diffusion model which then adds and subtracts noise in order to remove artefacts and smooth the entire output. In some embodiments, the diffusion model uses a U-Net architecture with attention mechanisms. In some embodiments, the CycleGAN and 3D CNN operate in an iterative, back-and-forth training process. In some embodiments, the method further comprises a dynamic weighting strategy for original 2D ultrasound, transformed, and synthetic images during model training. In accordance with a second aspect of the present disclosure, there is provided a system for implementing the method according to any of the preceding claims, comprising processors configured to execute the steps of the method. Further aspects may also provide a method for clinical use which may comprise the following: Collecting DICOM US Series Image from patient Passing DICOM US Series Image straight to the 3D architecture described which creates a 3D Model. This model may then be passed to Diffusion model which creates a smooth 3D model output which can be seen on screen and used for surgical planning, for point analysis (for exact stenosis readings), and for regression analysis (for fistula failure prediction). BRIEF DESCRIPTION OF THE DRAWINGS So that the manner in which the above recited features of the various embodiments can be understood in detail, more particular description of the inventive concepts, briefly summarized above, may be had by reference to various embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of the inventive concepts and are therefore not to be considered limiting of scope in anyway, and that there are other equally effective embodiments. Figure 1 is a flow diagram of method steps for analysing arteriovenous fistulas, AVFs, accordingto various embodiments; and Figures 2a and 2b provide a more detailed flow diagram of the method steps for analysing AVFs according to various embodiments In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to one of skill in the art that the inventive concepts may be practiced without one or more of these specific details. The description uses the example of capturing data related to an AVF and then generating a 3D image of an AVF. However, it will be understood that this is only an example and that the described steps could be applied to the imaging and / or analysis of any anatomical structure such as, for example, vascular structures (e.g carotid arteries), organs (e.g. kidneys and bony sites or bony landmarks) and anatomical landmarks. Figure 1 shows the training process based on the example of analysis of an AVF. Patients’ scans may be taken along the length of the fistula creating stacks of images. These images may be taken as 2D DICOM, Digital Imaging and Communications in Medicine. DICOM is a standardised ultrasound output. These stacks may be stored in a secure location, for example a server within the UK. The term stack is used to refer to a stack of images for one patients’ scan. Therefore, stack refers to one patients’ scan at one time. Next the obtained stacks (multiple scan stacks separated by individual or time) will be transformed mathematically. For this process the focus is on elastic and spatial transformations. 10 transformations may be created to be applied across the length of the scans. These transformations may be stored in a layer of annotation - referred to as T-layer, T-layers will only therefore appear on transformed images. Annotations may display the transformations at each image level and may describe co-ordinatal changes from the scan. Before selecting specific transformations, a small sample size of scans may be taken and trial different transformations at each level to create transformed stacks with believable and true to life anatomy. Once the selected transformations are suitable, the 6 selected transformations are applied. This process may create 10x the original scan number. Next these images may be fed into a diffusion model, this may allow for the creation of thousands of scan stacks. Creation may be validated at first by trained professionals, first 100, then out of the next 1000 10 in every 100 will be selected at random to be validated by experts in the field, thereafter 1 in every 100 will be passed to validator for rechecking to ensure consistency. In the next stage the diffusion model, the original scans, and the transformed scans may then be fed into a 2D CNN for annotation. For this a set number of image stacks at random (equal distribution across original, transformed and diffusion model stacks) may be taken and hand annotate each stack with information on anatomical information, the MLM may then complete the same task - comparing the outputs with the outputs from the hand annotated stacks. The iterative learning process takes place here and in a supervised way the MLM may attempt to find a way to repeat the results from the gold standard annotated stacks. The machine learning model may then once satisfying a set degree of accuracy may move onto validation stacks (another stack of equal size equally distributed from the remaining ultrasound stacks) the MLM may attempt to repeat the process in these - these may then be hand annotated again and comparisons made. Once the set degree of accuracy may be satisfied, we may then train on the remaining stacks. The output from this stage of training, may be a 2D CNN that may accurately annotate images with co-ordinate data of relevant anatomical intricacies of each scan. In the next stage 1000 hand crafted 3D models may be made, for example mesh made, with CAD. These 3D models may be selected at random from pools as a 2:1:1 ratio (original patient, transformed, diffusion) and Al generated and human generated 3D models may be compared (quantitively [with dice coefficients, etc] and qualitatively [expert analysis]). These may serve as test group - and the machine may run back and forth until achieving a set accuracy, importantly this accuracy may be based on coordinate data and resultant overall architecture of the model rather than fidelity. Concomitantly - the image stacks may be passed to the Cycle GAN to create 3D models in the same way - in this stage we may use the same process as above to create a 3D model. At this point we will compare the 2 3D models to one another, and hand over training may be started between CNN and CycleGAN where model creation may be started with one model and handed over to another and then back at discrete steps of creation (to be decided). A number of different stages off handover may be tested, it is thought by training by hand-over the resultant models may be high fidelity and more accurate than by one model alone. The chosen high-fidelity models may then be compared to the above single model cases. Importantly, the next stage of the process may be to denoise the 3D models using a 3D Diffusion architecture-this process may average the created models to prevent non-true to life and inaccurate fine details to appear. The method may comprise the following steps: 1. Diffusion model to prevent overfitting 2. 2D CNN for annotation 3. 3D CNN for model creation 4. Cycle GAN for fidelity improvement 5. Diffusion to smooth and average surface structure. The above works together in concert to create a high fidelity and smooth 3D model that may then be used for: prognosis / failure predictive data mining (data points created from annotation layer), Surgical planning 3D model, AR and VR with diffusion layer allowing the fistula to be viewed from every angle. Figure 1 illustrates a method (S100) for analysing arteriovenous fistulas, AVFs, data comprising the following steps: Receiving (S102) 2D ultrasound images of arteriovenous fistulas, AVFs; Generating (S104) transformed data by performing a 2D transformation and an elastic transformation with the 2D ultrasound images; Creating (S106) synthetic images by processing the 2D ultrasound data and the transformed data with a diffusion model; Converting (S108) 2D ultrasound images, transformed data and synthetic images into 3D models of AVFs using Cycle GAN; and Outputting (110) 3D models of AVFs. The method for analysing arteriovenous fistulas, AVFs, data comprises the following steps (as further illustrated in Figure 2a): Receiving 2D (S102) ultrasound images of arteriovenous fistulas, AVFs. 2D ultrasound images of arteriovenous fistulas (AVFs) as a dataset of raw 2D ultrasound scans are collected as initial input for the method. These images may be collected in a standardized DICOM format to ensure uniformity. Other image capture protocols may be used and normalisation techniques may be applied to standardise received images (to standardise resolution) and to reduce the noise in received images. These 2D ultrasound images of arteriovenous fistulas (AVFs) are unprocessed and contain crucial anatomical information that will be used for the subsequent steps. The 2D images can be passed through standard normalisation and noise reduction processes before annotation. This ensures the resolution is standardised and the noise is reduced. Annotation of the images may then be applied prior to analysis using 2D and 3D CNNs. The annotation may comprise one or more of the segmentation of the vessel wall in a first annotation layer. The application of doppler imaging principles in a second annotation layer. The incorporation of contextual information into the image in a third annotation layer. One or more of the annotation layers may be applied during annotation of the images. The method for analysing arteriovenous fistulas, AVFs, data further comprises the following step (as further illustrated in Figure 2a): Generating (S104) transformed data by performing a 2D transformation and an elastic transformation with the 2D ultrasound images. The raw 2D ultrasound images serve as the input for the 2 D transformation, The images are subjected to both 2D transformations and elastic transformations. The 2D transformation may be rotation and / or scaling. The elastic transformations may simulate non-rigid deformations like tissue stretching. During the 2D Transformation simple planar changes are applied to the images to alter their orientation or size without affecting their internal structure. The elastic transformation is more complex, non-linear changes are applied to mimic natural variations in tissue and blood vessel structures. The 2D transformation step produces a set of transformed images. Each original image may have multiple transformed versions, increasing the dataset size and variability. This expanded dataset may be critical for robust training in later stages. The transformation layer plays a crucial role in enhancing the dataset by introducing variability that mimics real-world conditions, which helps in makingthe machine learning models more robust. The transformation layer is a step in the data preprocessing pipeline where original 2D ultrasound images are systematically altered using a combination of 2D and elastic transformations. The transformation layer acts as a generator of additional data by creating new, altered versions of the original 2D ultrasound images. These transformed images are then used in training the models to improve their performance. There are different types of 2D transformations applied: rotation, scaling, translation and / or flipping. During rotation, the images are rotated by a certain degree to simulate different viewing angles. During scaling, the images are resized, either enlarged or reduced, to introduce variations in scale. During translation, the images are shifted horizontally or vertically, simulating changes in probe positioning during ultrasound scanning. During flipping, the images are flipped horizontally or vertically to introduce mirrored variations. There are different types of elastic transformations non-rigid deformations and / or grid distortion. During non-rigid deformations, elastic transformations apply deformations that stretch or compress parts of the image. This simulates natural variations in tissue and blood vessels, such as stretching or compression due to body movements or changes in blood pressure. During grid distortion, a grid is applied to the image, and the nodes of the grid are moved to create a warping effect, which distorts the image in a way that mimics real-world anatomical variations. The original 2D ultrasound images are fed into the transformation layer, where a set of predefined transformations is applied. Each image undergoes several different transformations, generating multiple versions of the same image. In the annotation layer, the transformations are annotated with metadata, which records the specific transformation applied to each image (e.g., degree of rotation, amount of scaling). This annotation layer is crucial for keeping track of the changes and understandingthe context of each transformed image during model training. Each transformation is applied uniformly to each set of serialised ultrasounds - meaning that a single ultrasound scan from a single patient at a single time creates a series of images (noted here as serialised ultrasounds, each ultrasound being a serial and each set of ultrasounds being a serialised set of ultrasounds) - gets multiple transformations but each transformation which is decided on and applied is importantly applied over the length of the series, there is weighting at this point as well to ensure that the scan has some variability. Furthermore, applying different transformations across a single series. This will allow for even greater variability of the outputs or whether these outputs will be too random to include in the next layer of input. Thereby, creatingtransformed image sets. Each original image may generate multiple transformed images, expanding the dataset significantly. For example, if 10 transformations are applied to each original image, the dataset size increases tenfold. The resulting transformed images are stored with their corresponding annotations in what is referred to as the T-layer, transformation layer. The T-layer is a specialized layer that keeps all the transformed images organized and annotated for later use in training. The images in the T-layer, along with their annotations, are used as input for training machine learning models like the CNN and CycleGAN. The transformed images introduce variations that the models learn to recognize and generalize, improving their ability to handle real-world data. During supervised learning, the models are trained in a supervised manner, where the original images and their transformations are labelled with corresponding anatomical features. This allows the model to learn how to correctly interpret and analyse both the original and transformed images. During data augmentation, the transformation layer effectively augments the dataset by creating numerous variations of each original image. This is particularly important in medical imaging, where collecting large datasets can be challenging. Thereby, creating an increased dataset size. The variability introduced by the transformations helps the models learn from a more diverse set of images, making them less sensitive to specific features of the original 2D ultrasound dataset. This diversity leads to better generalization when the models are applied to new, unseen data. Thereby, providing enhanced diversity. By exposing the models to transformed images that simulate real-world variations (like different angles, scales, and anatomical deformations), the models become more robust. They are better equipped to handle similar variations when processing actual patient data. Thereby, providing improved model robustness and handling variations. The use of transformed images helps reduce the risk of overfitting, where the model becomes too specialized to the original training 11 data and performs poorly on new data. The added variability forces the model to learn more generalized patterns rather than memorizing specific details. The elastic transformations, in particular, simulate non-rigid deformations that can occur naturally in the body. Training on these transformed images ensures that the models can accurately interpret scans that exhibit similar deformations, making the models more clinically relevant. Thereby, mimicking real-world conditions and achieving better clinical relevance. The increased variability in training data improves the models' ability to predict outcomes in different scenarios, such as detecting stenosis in AVFs that might be affected by various patient-specific factors. Thereby, enhancing predictive power. As, final output from training, the final models, trained using the original, transformed, and synthetic images, are more accurate and generalizable. They can better handle the wide range of variability encountered in clinical practice, leading to more reliable 3D reconstructions and better diagnostic support. In clinical applications, the transformation layer’s contribution ensures that the models are capable of providing high-quality 3D models that are useful in surgical planning, patient monitoring, and educational purposes. The robustness of these models directly translates into better patient outcomes and more efficient healthcare delivery. The transformation layer is a powerful tool in the machine learning pipeline, providing essential variability that enhances the model's robustness and generalizability. By systematically altering the original images and carefully annotating these changes, the transformation layer ensures that the models trained are not only accurate but also adaptable to a wide range of real-world clinical scenarios. That is to say, the 2D CNN is further trained on the standardised and annotated images. This enables feedback learningto be implemented on the 2D CNN to mitigate against model drift and to improve the accuracy of the model. Indeed, images may be anonymised to remove any data which may connect the image with a patient. The method for analysing arteriovenous fistulas, AVFs, data further comprises the following step (also further illustrated in Figure 2a): Creating (S106) synthetic images by processingthe 2D ultrasound data and the transformed data with a diffusion model. The transformed images from the previous step, along with the 2D ultrasound data as the original untransformed images, are inputted into a diffusion model. The diffusion model is a generative model that creates synthetic images by learning the distribution of the input data. The diffusion model iteratively generates new synthetic images that resemble 12 the original dataset but include variations introduced by the transformation process. These synthetic images are not mere copies but synthetic data that enriches the training set. The output from the diffusion model comprises a large number of synthetic images. These synthetic images are structurally similar to the original 2D ultrasound images but are generated artificially by the diffusion model. This synthetic dataset is essential for improving the training of the machine learning models, as it provides additional data points that reduce the risk of overfitting. The diffusion model serves two crucial functions in our invention: a) Creating synthetic images: The diffusion model may generate synthetic 2D ultrasound images by learning the distribution of the original and transformed images. It may operate by gradually adding noise to the input images and then learning to reverse this process. This allows to generate diverse, realistic ultrasound images that augment the training dataset. b) Denoising 3D models: After the 3D models are created, the diffusion model is employed to refine and denoise them. This process may involve treating the 3D model as a noisy input and iteratively applying the learned denoising process. This may smooth out artifacts and may improve the overall quality and accuracy of the 3D reconstructions. The diffusion model may use a U-Net architecture with attention mechanisms. Key parameters may include a noise schedule of 1000 steps, and a learning rate of 1 e-4 with the Adam optimizer. The diffusion model is a crucial component in the data augmentation and generation process, particularly in creating synthetic data that enhances the training of machine learning models. A diffusion model is a type of generative model used in machine learning to create synthetic data that closely resembles a given dataset. It works by learning the distribution of the original data and then generating new data points that follow this learned distribution. In the context of medical imaging, such as the arteriovenous fistula (AVF) ultrasound images, the diffusion model creates synthetic ultrasound images that are used to supplement the original dataset. The diffusion model is trained on the original and transformed 2D ultrasound images of AVFs. These are weighted. During training, the model learns the underlying patterns and features of these images, such as the structure of blood vessels and tissue textures. The diffusion model operates by adding noise to the original images in a controlled manner. This noise represents random variations that could exist in real-world data but are not present in the original dataset. The model gradually increases the noise level, moving the images towards a more uniform distribution for example Gaussian noise. Once the model has learned to add noise effectively, it then learns the reverse process—denoising. This involves starting from a noisy image and iteratively removing the noise to generate a synthetic image that resembles the original data. This reverse process is crucial because it allows the model to generate new, realistic images from random noise. In the generation phase, the model takes random noise as input and iteratively refines it to create new synthetic images that are statistically similar to the original 2D ultrasound images. The generation process involves multiple iterations, where at each step, the model applies a denoising process to bring the noisy image closer to a realistic ultrasound image. Each iteration reduces the noise level, gradually forming an image that resembles a plausible ultrasound scan. The final output of the method is a set of synthetic ultrasound images that closely mimic the structure and characteristics of the original 2D ultrasound images but are entirely generated by the model. These images are not direct copies but rather new examples that fit within the learned distribution of the original dataset. The diffusion model receives both the original and transformed ultrasound images as input. This diverse input allows the model to learn a broad distribution, encompassing both natural anatomical variations and those introduced by the transformations. In Data Augmentation, the synthetic images generated by the diffusion model are used to augment the original dataset. This expanded dataset includes real 2D ultrasound images, transformed images, and the newly generated synthetic images, all of which are used to train the machine learning models, CNN and CycleGAN. One of the primary benefits of the diffusion model is its ability to significantly increase the size of the dataset. Thereby, addressing data scarcity. In medical imaging, collecting large amounts of data can be challenging due to privacy concerns, costs, and the need for expert acquisition. The diffusion model helps overcome this limitation by generating additional data that can be used for training. The synthetic images introduce more variability into the training data. Thereby, providing diverse training data. This diversity helps the machine learning models learn to generalize better, as they are exposed to a wider range of examples. By learning to denoise and generate images from noise, the models trained with diffusion-generated data become more robust. They can better handle variations and anomalies in real-world data, leading to more reliable predictions and reconstructions. Since the synthetic images are not exact duplicates of the original 2D ultrasound data, they help prevent the models from overfitting. Overfitting occurs when a model becomes too specialized to the training data, performing poorly on new, unseen data. The introduction of synthetic data mitigates this risk by forcing the model to learn broader, more generalized patterns. The synthetic images often include subtle variations that might not be present in the original dataset. These variations help the model learn to interpret a wider range of scenarios, making it more adaptable to different patients and conditions. Thereby simulating real-world variability and providing enhanced generalization. The diffusion model’s ability to generate images from noise means it can produce examples of rare or atypical cases that might not be well-represented in the original 2D ultrasound dataset. This helps the model become more adept at recognizing and handling unusual or challenging cases. Thereby, anomalies are better handled. In the input to training, the final synthetic images generated by the diffusion model are combined with the original2D ultrasound and transformed images to create a comprehensive training set. This enriched dataset is used to train the CNN and CycleGAN models.: As weighting consideration, the synthetic images are given a lower weight during training compared to the original 2D ultrasound images. This ensures that while they contribute to the learning process, they do not dominate it, preserving the accuracy and reliability of the models. The models trained usingthis augmented dataset, including diffusion-generated images, are more robust, accurate, and generalizable. The 3D models as end product they produce from 2D ultrasound images are higher in quality, with improved fidelity and reliability, making them valuable for clinical applications such as surgical planning and diagnostic support. Unlike simple augmentation techniques (like flipping or rotating images), the diffusion model generates entirely new data points. These new images are more than just variations of existing ones; they are novel examples that enrich the training dataset. Thereby, achieving richness of generated data. The diffusion model can capture 15 and introduce complex variations in the data that are difficultto achieve with traditional augmentation methods. This includes subtle anatomical differences, variations in texture, and realistic noise patterns, all of which contribute to a more effective training process. Thereby, capturing complex variations. Once trained, the diffusion model can generate large volumes of synthetic data, which is particularly useful for training deep learning models that require vast amounts of data to achieve high performance. Thereby, generating large-scale data and high scalability. The diffusion model can be fine-tuned and updated as more original 2D ultrasound data becomes available, continuously improvingthe quality and realism of the synthetic data it generates. Thereby, achieving continuous improvement. The diffusion model is a powerful tool in the data generation process, providing synthetic data that significantly enhances the training of machine learning models. By simulating realistic variations and increasing the diversity of the dataset, it plays a crucial role in improving the accuracy, robustness, and generalizability of models used in medical imaging, particularly in the creation of 3D models from 2D ultrasound images. The method for analysing arteriovenous fistulas, AVFs, data further comprises the following step (as illustrated in Figure 2b): Converting (S108) 2D ultrasound images, transformed data and synthetic images into 3D models of AVFs using Cycle GAN. Three sets of images are used as input for the CNN (Convolutional neural network) and CycleGAN Integration: original 2D ultrasound images, transformed data, and synthetic images generated by the diffusion model. A Convolutional Neural Network (CNN) is trained using original 2D ultrasound images, transformed data, and synthetic images to recognize and annotate key anatomical features. The CNN focuses on identifying structures such as, for example, blood vessels and stenoses within the 2D images (i.e. the annotated and standardised versions of images captured from the patient). The CNN’s output includes annotated images where critical features are marked. These annotations are crucial for the next stages of the process. Marking critical features may comprise further processing where alphanumeric labels are added to the images identify the critical features with, for example, an identification number or a name, e.g. cervical stenosis or lumbar stenosis. The Cycle-Consistent Generative Adversarial Network, CycleGAN, is trained to convert 2D ultrasound images into 3D models. It learns by iteratively generating 3D images and comparing them back to 2D projections to ensure accuracy. The CycleGAN refines the 3D model generation by integrating feedback from the CNN annotations, leading to more accurate and realistic 3D models. The primary outputs here are refined 3D models generated from 2D ultrasound images. These models are improved versions, leveraging the annotations from the CNN and the iterative learning process of the CycleGAN. During the Feedback Training and Weighting of the 2D CNN, the inputs are the CNN-annotated images and the CycleGAN-generated 3D models. A feedback loop is established where the synthetic and transformed images are used alongside the original 2D ultrasound images to train the models continuously. The synthetic images are assigned lower weights compared to the original 2D ultrasound images to prevent overfitting to artificially generated data. This means that while the synthetic images are used to enhance training, they do not dominate the learning process. During weighting, the original 2D ultrasound images are given the highest weight, ensuring that the model’s learning is primarily based on real data. Transformed images are given moderate weight, and synthetic images are given the lowest weight. This balanced weighting ensures that the model learns from a diverse set of data while still prioritizing accuracy. The result and output is a set of highly trained models (CNN and CycleGAN) capable of accurately identifying and reconstructing AVFs from 2D ultrasound images. The feedback loop ensures continuous improvement in the model's accuracy and reliability. The final input includes the fully trained CNN and CycleGAN models, along with the original 2D ultrasound, transformed, and synthetic datasets. The final outputs are high-fidelity 3D models of AVFs. These models may be used for clinical applications, such as predicting potential fistula failures, planning surgeries, and providing visualizations for educational purposes. The models are robust, having been trained on a diverse dataset that includes real 2D ultrasound, transformed, and synthetic images. The 2D CNN for annotation may use a ResNet50 architecture pre-trained on ImageNet, with a final layer adapted for our specific annotation task. A learning rate of 1 e-5 with the Adam optimizer may be used. The 3D CNN may use a 3D U-Net architecture with 4 encoding and 4 decoding layers, each with 32, 64, 128, and 256 filters respectively. 3D convolutions with kernel size 3x3x3 and ReLU activation functions may be used. The CycleGAN may use two generators with a U-Net architecture and two discriminators using PatchGAN. A learning rate of 2e-4 and a batch size of 1 may be used due to memory constraints. The CycleGAN may play a crucial role in converting 2D DICOM images to 3D models. It may work in tandem with a 3D CNN architecture in an iterative, back-and-forth training process: 1. The CycleGAN may initially learn to map 2D DICOM image stacks to preliminary 3D representations. 2. These representations may be then passed to the 3D CNN, which may refine and may enhance the 3D structure. 3. The enhanced 3D structure may be then projected back to 2D space. 4. The CycleGAN may learn to minimize the difference between these projections and the original 2D images. 5. This process may be repeated, with each model improving based on the other's output. This cyclic training may allow for the generation of high-fidelity 3D models that maintain consistency with the original 2D data. This is step S110. For all models, we may use a combination of L1 loss, adversarial loss, and cycle consistency loss, with weights of 10,1, and 10 respectively. The concept of weightings refers to how different types of data (original 2 D ultrasound, transformed, and synthetic images) are prioritized during the training process of the machine learning models. Weighting is crucial to ensure that the models learn effectively from the data while avoiding overfitting to less reliable or less representative data. The original 2D ultrasound images are considered the most reliable source of data since they represent real patient anatomy. These images are given the highest weight duringtraining, meaning the model places the most importance on learning from this data. Prioritizing the original images helps the model to capture the most accurate and clinically relevant features, ensuring that the core learning is based on actual anatomical structures. Transformed images are created by applying 2D and elastic transformations to the original images. These transformations introduce variations that could occur naturally but are not 18 present in the original 2D ultrasound dataset. These transformed images are given a moderate weight. While these transformed images are not as reliable as the original images, they are valuable for introducing variability into the training process. This variability helps the model generalize better, making it more robust when dealing with different patient data. The synthetic images generated by the diffusion model are assigned the lowest weight. These images are entirely artificial, created by the model based on learned patterns rather than directly derived from real-world scans. Although synthetic images expand the dataset and add diversity, they are less trustworthy since they might introduce artifacts or unrealistic features. Therefore, they are used cautiously to avoid the model overfitting to these potentially less accurate representations. During the training phase of both the CNN and CycleGAN models, the images are fed into the models in batches. Each batch includes a mix of original, transformed, and synthetic images. As weighting mechanism, the weightings are applied such that the influence of each image on the model’s learning process corresponds to its assigned weight. For example, during the backpropagation phase where the model adjusts its parameters based on errors in prediction, the errors from original images will have a more significant impact on updating the model’s weights compared to those from synthetic images. Since the original 2D ultrasound images have the highest weight, they have the most influence on the model’s parameter updates. The model’s ability to identify and reconstruct key features is predominantly shaped by the original 2D ultrasound data, ensuring that the model remains accurate and clinically relevant. The transformed images are moderately weighted images help the model learn to handle variations and anomalies. They introduce flexibility into the model’s learning, allowing it to generalize better when applied to new, unseen data. While the synthetic images have the lowest weight, synthetic images still contribute by enhancing the dataset’s diversity. This helps in avoiding overfitting to the specific characteristics of the original 2D ultrasound dataset. However, the lower weight of the synthetic images ensures that any potential errors introduced by synthetic data do not significantly skew the model’s learning. At the start of training, the original 2D ultrasound images may have even higher relative weight to ensure that the model first learns to accurately recognize and reconstruct the most critical features. As training progresses, the weight of transformed and synthetic images can be slightly increased to allow the model to learn from these additional data sources without 19 deviating from the accuracy established by the original 2D ultrasound images. In the later stages of training, the weightings might be fine-tuned based on the model’s performance. If the model shows signs of overfitting to the original data, the weight of transformed and synthetic images might be adjusted to introduce more diversity into the learning process. Thereby, dynamically adjusting of weightings. With this weighting strategy overfitting is prevented, generalization is enhanced, and accuracy is maintained. By giving lower weight to synthetic images, the model avoids overfitting to data that might contain unrealistic features. This ensures that the model remains generalizable and performs well on real-world data. Thereby, preventing overfitting. The use of moderately weighted transformed images helps the model become robust to variations, such as different patient anatomies or slight differences in ultrasound acquisition techniques. The high weight assigned to original images ensures that the model’s predictions and reconstructions are rooted in real, accurate data, maintaining the clinical relevance of the output. Thereby, maintaining accuracy. The final output of this training process, considering the applied weightings, is a highly accurate and generalizable model capable of creating reliable 3D reconstructions of AVFs from 2D ultrasound images. The model effectively balances learning from authentic, varied, and artificially generated data, making it robust and clinically useful. The weighting strategy described in the document may be sound. However, it could be improved by implementing a dynamic weighting scheme: A dynamic weighting strategy may be used where the weights assigned to original, transformed, and synthetic images may evolve during the training process. Initially, original images may be given the highest weight (e.g., 0.6), transformed images a moderate weight (e.g., 0.3), and synthetic images the lowest (e.g., 0.1). As training progresses, the weight of transformed and synthetic images is gradually increased, based on the model's performance on a validation set. This allows the model to learn core features from original data first, then generalize better using the augmented data. It should be noted that the above-mentioned aspects and embodiments illustrate rather than limit the disclosure, and that those skilled in the art will be capable of designing many alternative embodiments without departing from the scope of the disclosure as defined by the appended claims. In the claims, any reference signs placed in parentheses shall not be construed as limiting the claims. The word "comprising" and "comprises", and the like, does not exclude the presence of elements or steps other than those listed in any claim or the specification as a whole. In the present specification, “comprises” means “includes or consists of” and “comprising” means “including or consisting of”. The singular reference of an element does not exclude the plural reference of such elements and vice-versa. The disclosure may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. Method for analysing at least one anatomical structure, the method implemented by a processing resource, the method comprising the following steps:Receiving 2D ultrasound images of the at least one anatomical structure;Generating transformed data by performing a 2D transformation and an elastic transformation with the 2D ultrasound images;Creating synthetic images by processing the 2D ultrasound data and the transformed data with a diffusion model;Converting 2D ultrasound images, transformed data and synthetic images into 3D models of the at least one anatomical structure using Cycle GAN; and Outputting 3D models of the at least one anatomical structure.

2. Method accordingto claim 1, wherein the 2D ultrasound images are in a standardized DICOM format, optionally wherein the 2D ultrasound images are original DICOM US Series Images, DUSI.

3. Method accordingto any of the preceding claims further comprising storingthe 2D ultrasound images.

4. Method accordingto any of the preceding claims further comprising using a 2D CNN to annotate the 2D ultrasound images to show exact vessel anatomical positions.

5. Method accordingto any of the preceding claims further comprising transforming the 2D ultrasound images with elastic and set transformations, optionally storing the transformed DICOM US Series Images, TDUSI.

6. Method accordingto claim 5 further comprising annotating the 2D ultrasound images with a 2D Convolutional Neural Networks, CNN.

7. Method accordingto any of the preceding claims further comprising comparing TDUSI with original 2D ultrasound images, optionally assuring quality through independent operator (human) analysis.

8. Method according to any of the preceding claims further comprising passing DUSI and TDUSI into a diffusion model, where the computer learns to create similar images, and weighting with DUSI higher weights and TDUSI lower weights.

9. Method according to any of the preceding claims further comprising completing the diffusion model training and outputting Diffusion Model DICOM US Series Images, DMDUSI, optionally the 2D CNN is used to annotate the DMDUSI.

10. Method accordingto any of the preceding claims further comprising passing DUSI, TDUSI and DMDUSI for pass back training between Cycle GAN and 3D CNN, optionally weighting with highest, TDUSI middle and DMDUSI lowest, as a weighting strategy.

11. Method accordingto claim 10 further comprising using the weighting strategy throughout training to dynamically alter weights of training.

12. Method accordingto any of the preceding claims further comprising completing the 3D Model training and outputting 3D models.

13. Method accordingto any of the preceding claims further comprising passing the 3D Models to another diffusion model which then adds and subtracts noise in order to remove artefacts and smooth the entire output.

14. Method accordingto any of the preceding claims, wherein the diffusion model uses a U-Net architecture with attention mechanisms.

15. Method accordingto any of the preceding claims, wherein the CycleGAN and 3D CNN operate in an iterative, back-and-forth training process.

16. Method according to any of the preceding claims further comprising a dynamic weighting strategy for original 2D ultrasound, transformed, and synthetic images during model training.

17. A system for implementing the method according to any of the preceding claims, comprising processors configured to execute the steps of the method.Application No: GB2416877.5 Examiner: Kalim YasseenClaims searched: 1-17 Date of search: 19 May 2025Patents Act 1977: Search Report under Section 17Documents considered to be relevant:Category Relevant to claims Identity of document and passage or figure of particular relevance Y 1-17 US2023 / 0281842 Al (RIBEIRO) see whole document especially paragraphs 23, 31. 37, 38 &149 Y 1-17 CN117611701 A (UNIV CHONGQING TECHNOLOGY) see whole document especially Background technique', WPI abstract A - US2022 / 0101048 Al (TAN) A - US2021 / 0174938 Al (PARK)Categories:X Document indicating lack of novelty or inventive step A Document indicating technological background and / or state of the art. Y Document indicating lack of inventive step if combined with one or more other documents of same category. P Document published on or after the declared priority date but before the filing date of this invention. & Member of the same patent family E Patent document published on or after, but with priority date earlier than, the filing date of this application.Field of Search:Search of GB, EP, WO &US patent documents classified in the following areas of the UKCX :Worldwide search of patent documents classified in the following areas of the IPC_____________G06F; G06T____________________________________________________The following online and other databases have been used in the preparation of this search reportSEARCH-PATENT, SEARCH-NPLInternational Classification:Subclass Subgroup Valid From G16H 0030 / 20 01 / 01 / 2018 G06T 0007 / 00 01 / 01 / 2017 G06T 0011 / 00 01 / 01 / 2006 G06T 0015 / 00 01 / 01 / 2011 G16H 0030 / 40 01 / 01 / 2018 G16H 0050 / 20 01 / 01 / 2018

Images