Method and system for training a machine learning model to predict abdominal aortic aneurysm (AAA) growth in images.

Abstract

A method and system are provided for predicting AAA growth based on at least one image of a given patient previously diagnosed with abdominal aortic aneurysm AAA, using one or more machine learning (ML) models. Baseline and follow-up images of the patient with AAA are received and compared, and the difference in the aortic region is calculated. Each baseline image is labeled as showing a significant or non-significant AAA growth based on the calculated difference. Features are extracted from the aortic region of the baseline images, and the features include one or more of shape features, texture features, or deep features. The ML model is trained to classify the baseline images as showing or not showing a significant AAA growth based on the extracted features. The amount of calcification accumulation in the baseline images is also calculated as an indicator of AAA growth.

Description

Technical Field

[0001] This technology relates to the field of medical imaging. More specifically, this technology relates to methods and systems for training a machine learning (ML) model and using the ML model to predict the growth of abdominal aortic aneurysms (AAA) in images.

Background Art

[0002] An abdominal aortic aneurysm (AAA) is defined as a fusiform dilation of the aorta and, if left undiagnosed and untreated, will progress and rupture. Aneurysms with a diameter exceeding 5 - 5.4 cm are considered at risk of rupture, but the unexpected rupture risk of aneurysms with a diameter less than 5 cm, estimated at 5% per year, indicates the contribution of other factors in AAA growth. In fact, using diameter as the sole determinant for evaluating AAA growth is not only inadequate but also inaccurate. The reason is that the growth rate is a patient-specific factor that varies over time for each individual.

[0003] Remodeling of the arterial wall, abnormal blood patterns and recirculation within the AAA sac, and the formation of intraluminal thrombus (ILT) are other important factors to consider when investigating the progression of AAA.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The purpose of this technology is to improve at least some of the inconveniences present in the prior art. One or more implementations of this technology may give and / or broaden the scope of methods and / or approaches to achieving the goals and objectives of this technology. [Means for solving the problem]

[0006] The developers of this technology realized that machine learning techniques could be used to identify image-based parameters that correlate with aneurysm enlargement in images.

[0007] The developers of this technology propose investigating its role as an indicator of increased AAA (Advanced Computational Index), including lumen shape and lumen contrast in images.

[0008] The developers recognized that lumen shape and lumen contrast are central to the development of hemodynamics and flow patterns. Referring to Figure 1A, the first CT image 20 of a patient diagnosed with AAA and showing a considerable degree of AAA enlargement is shown, and it can be seen that the lumen 22 deviates from a circular shape.

[0009] The developers also recognized that heterogeneity of luminal contrast transmits information about blood flow patterns and indicates areas of slow flow and / or turbulence related to the inflammatory mechanism of the wall. Referring to Figure 1B, a second CT image 30 of a patient's body diagnosed with AAA and showing a considerable degree of AAA enlargement is shown, and heterogeneity of contrast in the lumen 32 can be seen to represent turbulent blood flow.

[0010] Furthermore, the presence of calcification in the ILT and wall of the aneurysm can lead to further damage to the wall.

[0011] The developers of this technology propose using a machine learning model to investigate the contribution of features extracted from the lumen, ILT, and calcification to the prediction of AAA increase.

[0012] One or more implementations of this technology enable improved growth prediction by incorporating image information regarding the lumen, ILT, and calcification. One or more implementations of this technology propose using features extracted from images as a tool to investigate flow turbulence in the aorta.

[0013] Therefore, one or more implementations of this technology relate to methods and systems that train and use machine learning models to predict abdominal aortic aneurysm (AAA) growth in images based on their features.

[0014] In one or more alternative implementations of this technology, the method, system, and non-temporary storage medium may be adapted and used to predict the growth of other types of aneurysms within blood vessels, such as thoracic aneurysms.

[0015] According to a broad aspect of the present technology, a method is provided for training at least one classifier to predict aneurysm enlargement in images acquired by a medical imaging device, the method being performed by at least one processor. The method includes the steps of receiving, for each of a plurality of patients, a set of baseline images and a set of follow-up images acquired by a medical imaging device in different imaging sessions, each patient being diagnosed with an aneurysm, and comparing the set of baseline images and the set of follow-up images, and determining the respective differences in the vascular region for each respective subset of the baseline images and the associated subset of the follow-up images. The method further includes the step of labeling each subset of the baseline images with a significant aneurysm enlargement label in response to the respective difference in the aortic region exceeding a threshold. The method further includes a training step of assigning a non-significant aneurysm enlargement label to each subset of baseline images and associated subsets of follow-up images in response to each difference in the aortic region being below a threshold; extracting each set of features from the aortic region for each subset of baseline images; training at least one classifier to classify sets of baseline images based on each set of features by using each label as a target, wherein for each baseline image, at least one classifier is used to classify each subset of baseline images as indicating either significant or non-significant aneurysm enlargement based on the set of features, and to obtain each prediction of aneurysm enlargement; and updating at least one parameter of at least one classifier based on each prediction and each label; and outputting a trained classifier with the updated parameter.

[0016] According to a broad aspect of the present technology, a method is provided for training at least one classifier to predict abdominal aortic aneurysm (AAA) enlargement in images acquired by a medical imaging device, the method being performed by at least one processor. The method comprises the steps of: receiving a set of baseline images and a set of follow-up images acquired by a medical imaging device in different imaging sessions for each of a group of patients, each patient being diagnosed with AAA; comparing the set of baseline images and the set of follow-up images; determining the respective differences in the aortic region for each subset of the baseline images and the associated subset of the follow-up images; labeling each subset of the baseline images with a significant AAA enlargement in response to the respective differences in the aortic region exceeding a threshold; and labeling each subset of the baseline images with a significant AAA enlargement in response to the respective differences in the aortic region falling below a threshold. The training step includes the steps of: labeling each non-significant AAA increase; extracting each set of features from the aortic region for each subset of baseline images; training at least one classifier to classify sets of baseline images based on each set of features by using each label as a target, wherein for each baseline image, at least one classifier is used to classify each subset of baseline images as exhibiting either significant or non-significant AAA increase based on the set of features, and obtaining each prediction of AAA increase; and updating at least one parameter of at least one classifier based on each prediction and each label; and outputting a trained classifier with the updated parameter.

[0017] In one or more implementations of the present method, each set of feature quantities includes shape feature quantities extracted from the lumen in the baseline image.

[0018] In one or more implementations of the present method, the shape feature quantities include histogram of oriented gradients (HOG) feature quantities.

[0019] In one or more implementations of the present method, each set of feature quantities includes texture feature quantities indicating the contrast extracted from the lumen in the baseline image.

[0020] In one or more implementations of the present method, the texture feature quantities include gray-level co-occurrence matrix (GLCM) feature quantities.

[0021] In one or more implementations of the present method, for each baseline image, the step of extracting each set of feature quantities includes extracting deep feature quantities from the lumen in the image collected by the medical imaging device using at least one feature quantity extractor.

[0022] In one or more implementations of the present method, at least one feature quantity extractor is configured to extract deep feature quantities from the lumen, the blood vessel wall, and intraluminal thrombus (ILT).

[0023] In one or more implementations of the present method, at least the feature quantity extractor includes a convolutional neural network (CNN).

[0024] In one or more implementations of the present method, at least one classifier includes an ensemble tree.

[0025] In one or more implementations of the method, each subset of the baseline images includes a plurality of baseline images, and each associated subset of the follow-up images each includes a plurality of associated follow-up images.

[0026] In one or more implementations of the method, each difference in the aortic region includes the differences in the aortic region parallel to the cross-section and the aortic region parallel to the sagittal plane.

[0027] According to a broad aspect of the present technology, there is provided a method for predicting a predicted increase in an aneurysm based on at least one image of a given patient previously diagnosed with an aneurysm, the method being executed by at least one processor, the processor having access to a trained classifier trained to classify an image of a patient as indicative of aneurysm growth or not indicative of aneurysm growth. The method includes receiving a set of images of a body including the aorta of a given patient, the set of images including at least one image, the set of images having been collected using a medical imaging device; segmenting each of the set of images using at least one trained segmentation model to extract a vascular region including the lumen of the given patient; extracting a set of features including lumen shape features indicative of the shape of the lumen from the lumen; using the trained classifier to classify each subset of the set of images as indicative of aneurysm growth or not indicative of aneurysm growth based on at least the set of features to obtain a classified set of images for the given patient; predicting whether aneurysm growth is seen in the patient based on at least the classified set of images; and outputting the prediction.

[0028] In a broad aspect of the present technology, a method is provided for predicting AAA growth based on at least one image of a given patient previously diagnosed with abdominal aortic aneurysm (AAA), the method being performed by at least one processor, the processor having access to a trained classifier trained to classify patient images as either indicating or not indicating AAA growth. The method includes receiving a set of images of a given patient's body, including the aorta, the set of images comprising at least one image, the set of images being collected using a medical imaging device; segmenting each of the set of images using at least one trained segmentation model to extract the aortic region including the lumen of the given patient; extracting a set of features from the lumen, including lumen shape features indicating the shape of the lumen; classifying each subset of the set of images as either indicating or not indicating AAA growth based on at least the set of features using a trained classifier to obtain a set of classified images for the given patient; predicting whether the patient will have AAA growth based on at least the classified set of images; and outputting the prediction.

[0029] In one or more implementations of the method, the method further includes the steps of: segmenting calcification in each aortic region of a set of images using at least one trained segmentation model; calculating the amount of calcification in each of the classified sets of images and determining whether the amount of calcification exceeds a threshold; and, in response to the amount of calcification exceeding the threshold, outputting the amount of calcification as a further indicator of AAA increase for a given patient.

[0030] In one or more implementations of this method, the set of features includes a set of lumen texture features that represent the contrast of the lumen of a given patient, and the classification step is further based on the set of lumen texture features.

[0031] In one or more implementations of this method, the set of luminal shape features includes gradient direction histogram (HOG) features.

[0032] In one or more implementations of this method, the set of luminal texture features includes gray-level co-occurrence matrix (GLCM) features.

[0033] In one or more implementations of this method, the step of extracting a set of features from the lumen is performed by a feature extraction machine learning (ML) model, and the set of features corresponds to deep features.

[0034] In one or more implementations of this method, the trained classifier includes an ensemble tree.

[0035] According to a broad aspect of the present technology, a system is provided for training at least one classifier to predict the enlargement of aneurysms in images acquired by a medical imaging device, the system comprising a non-temporary computer-readable medium for storing instructions, and at least one processor operably connected to the non-temporary computer-readable medium. The at least one processor, upon executing an instruction, receives a set of baseline images and a set of follow-up images for each of a plurality of patients, each patient being diagnosed with an aneurysm, compares the received set of baseline images and the set of follow-up images, calculates the respective difference in the vascular region for each subset of the baseline images and the associated subset of the follow-up images, labels each subset of the baseline images with a significant aneurysm enlargement in response to each difference in the vascular region exceeding a threshold, and labels each subset of the baseline images with a significant aneurysm enlargement in response to each subset of the baseline images and the associated subset of the follow-up images with a respective difference in the vascular region below a threshold. The method comprises the steps of: labeling each subset with a non-significant aneurysm enlargement label; extracting each set of features from the vascular region for each subset of baseline images; training at least one classifier to classify the set of baseline images based on each set of features by using each label as a target, wherein for each baseline image, the classifier is used to classify each subset of baseline images as either significant or non-significant aneurysm enlargement based on the set of features, obtaining each prediction; and updating at least one parameter of the classifier based on each prediction and each label; and outputting a trained classifier with the updated parameter.

[0036] According to a broad aspect of this technology, a system is provided for training at least one classifier to predict abdominal aortic aneurysm (AAA) growth in images acquired by a medical imaging device. The system comprises a non-temporary computer-readable medium for storing instructions and at least one processor operably connected to the non-temporary computer-readable medium, wherein, upon executing an instruction, the at least one processor receives a set of baseline images and a set of follow-up images acquired by a medical imaging device in different imaging sessions for each of a plurality of patients, each patient being diagnosed with AAA, and compares the received set of baseline images and the set of follow-up images, calculates the respective difference in the aortic region for each subset of the baseline images and the associated subset of the follow-up images, and labels each subset of the baseline images with a significant AAA growth in response to each difference in the aortic region exceeding a threshold, and labels each subset of the baseline images and the associated subset of the follow-up images with a significant AAA growth in the aortic region The method comprises training a classifier to classify a set of baseline images based on each set of features, using each label as a target, in response to each difference falling below a threshold; extracting each set of features from the aortic region for each subset of baseline images; and training a classifier to classify a set of baseline images based on each set of features, wherein for each baseline image, the classifier is used to classify each subset of baseline images based on the set of features as either significant or non-significant AAA increase, obtaining each prediction of AAA increase; and updating at least one parameter of the classifier based on each prediction and each label, and outputting a trained classifier with the updated parameter.

[0037] In one or more implementations of this system, each set of features includes shape features extracted from the lumen in the baseline image.

[0038] In one or more implementations of this system, the shape features include gradient direction histogram (HOG) features.

[0039] In one or more implementations of this system, each set of features includes texture features that represent contrast extracted from the lumen in the baseline image.

[0040] In one or more implementations of this system, texture features include gray-level co-occurrence matrix (GLCM) features.

[0041] In one or more implementations of this system, extracting each set of features for each baseline image includes using at least one feature extractor to extract deep features from the lumen in the image collected by the medical imaging device.

[0042] In one or more implementations of this system, at least one feature extractor is configured to extract deep features from the lumen, vessel wall, and intraluminal thrombus (ILT).

[0043] In one or more implementations of this system, the feature extractor includes at least a convolutional neural network (CNN).

[0044] In one or more implementations of this system, at least one classifier includes an ensemble tree.

[0045] In one or more implementations of this system, each subset of baseline images contains multiple baseline images, and each associated subset of follow-up images contains multiple associated follow-up images.

[0046] In one or more implementations of this system, each difference in the aortic region includes the differences in the aortic region parallel to the transverse plane and the aortic region parallel to the sagittal plane.

[0047] According to a broad aspect of the present technology, a system is provided for predicting aneurysm growth based on at least one image of a given patient previously diagnosed with an aneurysm, the system comprising a non-temporary computer-readable medium for storing instructions and at least one processor operably connected to the non-temporary computer-readable medium, the at least one processor having access to a trained classifier trained to classify the patient's image as either indicating aneurysm growth or not indicating aneurysm growth. At least one processor is configured to, upon executing an instruction, receive a set of images of a given patient's body, including the aorta, the set of images including at least one image, the set of images being collected using a medical imaging device, segment each of the set of images using at least one trained segmentation model to extract vascular regions including the lumen of the given patient, extract a set of features from the lumen including lumen shape features indicating the shape of the lumen, classify each subset of the set of images using at least a set of features to indicate or not indicate aneurysm enlargement using a trained classifier to obtain a set of classified images for the given patient, predict whether the patient has aneurysm enlargement based on at least the set of classified images, and output the prediction.

[0048] According to a broad aspect of the present technology, a system is provided for predicting AAA growth based on at least one image of a given patient previously diagnosed with an abdominal aortic aneurysm (AAA). The system comprises a non-temporary computer-readable medium for storing instructions and at least one processor operably connected to the non-temporary computer-readable medium, the at least one processor having access to a trained classifier trained to classify the patient's images as either indicating AAA growth or not indicating AAA growth. At least one processor is configured to, upon executing an instruction, receive a set of images of a given patient's body, including the aorta, the set of images including at least one image, the set of images being collected using a medical imaging device, segment each of the set of images using at least one trained segmentation model to extract the aortic region including the lumen of the given patient, extract from the lumen a set of features including lumen shape features indicating the shape of the lumen, classify each subset of the set of images using at least the set of features as either showing AAA enlargement or not showing AAA enlargement, obtain a set of classified images for the given patient, predict whether the patient has AAA enlargement based on at least the set of classified images, and output the prediction.

[0049] In one or more implementations of this system, at least one processor is further configured to segment calcification in each aortic region of a set of images using at least one trained segmentation model, calculate the amount of calcification in each of the classified sets of images, determine whether the amount of calcification exceeds a threshold, and, in response to the amount of calcification exceeding the threshold, output the amount of calcification as a further indicator of AAA increase for a given patient.

[0050] In one or more implementations of this system, the set of features includes a set of lumen texture features that represent the contrast of the lumen of a given patient, and the classification is further based on the set of lumen texture features.

[0051] In one or more implementations of this system, the set of luminal shape features includes gradient direction histogram (HOG) features.

[0052] In one or more implementations of this system, the set of luminal texture features includes gray-level co-occurrence matrix (GLCM) features.

[0053] In one or more implementations of this system, the extraction of a set of features from the lumen is performed by a feature extraction machine learning (ML) model, and the set of features corresponds to deep features.

[0054] In one or more implementations of this system, the trained classifier includes an ensemble tree.

[0055] Terms and Definitions In the context of this specification, “server” is a computer program that runs on appropriate hardware and is capable of receiving requests (e.g., from electronic devices) via a network (e.g., a communication network), and executing or causing such requests to be executed. Hardware may be a single physical computer or a single physical computer system, but does not have to be either for the purposes of this technology. In this context, the use of the expression “server” does not mean that all tasks (e.g., received commands or requests) or any particular task are received, executed, or caused to be executed by the same server (i.e., the same software and / or hardware), but rather that any number of software elements or hardware devices may receive / send, execute, or cause to be executed any task or request, or be involved in the results of any task or request, and all of this software and hardware may be one server or more servers, both of which are included in the expressions “at least one server” and “server.”

[0056] In the context of this specification, “computing device” is any computing device or computer hardware capable of running software appropriate for the task at hand. Therefore, some (non-exclusive) examples of electronic devices include general-purpose personal computers (desktops, laptops, netbooks, etc.), mobile computing devices, smartphones, and tablets, as well as network devices such as routers, switches, and gateways. Note that electronic devices in this context are not excluded from acting as servers for other electronic devices. The use of the term “computing device” does not exclude multiple computing devices used to receive / transmit, execute, or cause to execute any task or request, or the results of any task or request, or any step in any method described herein. In the context of this specification, “client device” refers to any of the various end-user client computing devices associated with a user, such as personal computers, tablets, and smartphones.

[0057] In the context of this specification, unless otherwise explicitly stated, computer systems may refer to, but are not limited to, “electronic devices,” “computing devices,” “operating systems,” “systems,” “computer-based systems,” “computer systems,” “network systems,” “network devices,” “controller units,” “monitoring devices,” “control devices,” “servers,” and / or any combination thereof appropriate for the task at hand.

[0058] In the context of this specification, the term “computer-readable storage medium” (also referred to as “storage medium” and “storage”) includes, but is not limited to, any non-temporary medium of any nature and type, including RAM, ROM, disks (such as CD-ROMs, DVDs, floppy disks, and hard drives), USB keys, solid-state drives, tape drives, etc. Multiple components may be combined to form a computer information storage medium, such as two or more media components of the same type and / or two or more media components of different types.

[0059] In the context of this specification, “database” is any structured collection of data, independent of its specific structure, database management software, or computer hardware on which the data is stored, implemented, or otherwise made available for use. A database may reside on the same hardware as the processes that store or utilize the information stored in it, or it may reside on separate hardware, such as a dedicated server or multiple servers.

[0060] In the context of this specification, the term "information" includes any nature or type of information that can be stored in a database. Therefore, information includes, but is not limited to, audiovisual works (images, videos, audio recordings, presentations, etc.), data (location data, numerical data, etc.), text (opinions, comments, questions, messages, etc.), documents, spreadsheets, word lists, etc.

[0061] In the context of this specification, unless otherwise explicitly stated, the “reference” of an information element may be the information element itself, or a pointer, reference, link, or other indirect mechanism that enables the recipient of the reference to identify the location of a network, memory, database, or other computer-readable medium from which the information element may be retrieved. For example, a reference to a document may include the document itself (i.e., its contents), or a reference to a document may be a unique document descriptor that identifies a file for a particular file system, or some other means that directs the recipient of the reference to a network location, memory address, database table, or any other location from which the file may be accessed. As a person skilled in the art will recognize, the precision required in such a reference depends on the degree of prior understanding of the interpretation to be given to the information exchanged, such as between the sender and receiver of the reference. For example, if it is understood before communication between the sender and receiver that the instruction for an information element takes the form of a database key for an entry in a specific table of a given database containing the information element, then even if the information element itself is not transmitted between the sender and receiver of the instruction, only the transmission of the database key is required to effectively convey the information element to the receiver.

[0062] In the context of this specification, the term “communication network” includes telecommunication networks such as computer networks, the Internet, telephone networks, Telex networks, TCP / IP data networks (e.g., WAN networks, LAN networks, etc.). The term “communication network” includes wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media, as well as any combination thereof.

[0063] In the context of this specification, words such as “first,” “second,” and “third” are used as adjectives solely to allow for distinction between the nouns they modify and not to describe any particular relationship between those nouns. Therefore, it should be understood that the use of terms such as “first server” and “third server” does not imply any particular order, type, chronological order, hierarchy, or ranking of servers / between servers, nor does their use (by itself) imply that a “second server” must necessarily exist in a given situation. Furthermore, as described in other contexts of this specification, references to “first” and “second” elements do not exclude the two elements from being the same actual, real-world elements. Therefore, for example, in some cases the “first” server and the “second” server may have the same software and / or hardware, while in other cases they may have different software and / or hardware.

[0064] Each implementation example of this technology has at least one of the above-described objectives and / or embodiments, but not necessarily all of them. It should be understood that some embodiments of this technology resulting from attempts to achieve the above-described objectives may not satisfy these objectives and / or may satisfy other objectives not specifically described herein.

[0065] Additional and / or alternative features, embodiments, and advantages of implementations of this technology will become apparent from the following description, the accompanying drawings, and the accompanying claims.

[0066] For a deeper understanding of this technology, as well as other embodiments and further features thereof, please refer to the following description, which should be used in conjunction with the accompanying drawings. [Brief explanation of the drawing]

[0067] [Figure 1A] This figure shows the first CT image in a patient with a considerable degree of AAA enlargement, in which luminal shape deviation is observed. [Figure 1B] This figure shows a second CT image in a patient with a considerable degree of AAA enlargement, where heterogeneity of luminal contrast is observed. [Figure 2] This is a schematic diagram of an electronic device based on one or more non-exclusive implementation examples of this technology. [Figure 3] This is a schematic diagram of a communication system based on one or more non-exclusive implementations of this technology. [Figure 4] This is a schematic diagram of the AAA growth prediction training procedure using one or more non-exclusive implementations of this technology. [Figure 5] This figure shows the input and output of a feature extraction procedure using one or more non-restrictive implementations of this technology. [Figure 6] This figure shows the input and output of alignment and comparison procedures in one or more non-exclusive implementation examples of this technology. [Figure 7] This figure shows the training procedure using one or more non-exclusive implementations of this technology. [Figure 8] This chart shows the amount of calcification accumulation corresponding to the patient number, comparing calcification in slices labeled as significantly increased and slices labeled as not significantly increased for each patient, using one or more non-exclusive implementations of this technology. [Figure 9] This is a plot of the test classification error, corresponding to the number of trees acquired during the training procedure, for one or more non-restrictive implementations of this technology. [Figure 10] This is a flowchart of a method for training at least one machine learning (ML) model to predict abdominal aortic aneurysm (AAA) growth, the method being performed according to one or more non-limiting implementations of the present technique. [Figure 11] This is a flowchart of a method for predicting AAA growth in patients diagnosed with abdominal aortic aneurysm (AAA) using a machine learning (ML) model, the method being performed according to one or more non-exclusive implementations of this technique. [Modes for carrying out the invention]

[0068] The examples and conditional statements described herein are primarily intended to help the reader understand the principles of this technology, and do not limit the scope of this technology to such specifically described examples and conditions. Those skilled in the art will understand that various configurations not expressly described or illustrated herein can still be devised to embody the principles of this technology and fall within its intent and scope.

[0069] Furthermore, for the sake of understanding, the following description may illustrate a relatively simplified implementation of this technology. As those skilled in the art will understand, various implementations of this technology can be more complex.

[0070] In some cases, examples of modifications to the Technology may be included, which may be considered useful. This is done solely for the purpose of aiding understanding and, again, not to limit the scope of the Technology or to describe its limitations. These modifications are not an exhaustive list, and a person skilled in the art may still make other modifications while remaining within the scope of the Technology. Furthermore, where no examples of modifications are provided, modifications should not be construed as impossible and / or as the described modifications being the only way to implement that element of the Technology.

[0071] Furthermore, all sentences in this specification describing the principles, embodiments, and implementations of the present technology, as well as specific examples thereof, encompass both their structural and functional equivalents, whether currently known or to be developed in the future. Therefore, for example, any block diagram in this specification will be understood by those skilled in the art to represent a conceptual diagram of an exemplary circuit embodying the principles of the present technology. Similarly, any flowchart, flow diagram, state transition diagram, pseudocode, etc., can be represented in substantially computer-readable media and will therefore be understood to represent various processes that can be performed by such a computer or processor, whether or not such a computer or processor is explicitly illustrated.

[0072] The functionality of the various elements shown in the diagram, including functional blocks labeled "processor" or "graphics processing unit," can be provided by dedicated hardware, as well as hardware capable of running software in conjunction with appropriate software. When provided by a processor, functionality can be provided by a single dedicated processor, a single shared processor, or by multiple individual processors, some of which may be shared. In some non-exclusive implementations of this technology, the processor may be a general-purpose processor, such as a central processing unit (CPU), or a processor dedicated to a specific purpose, such as a graphics processing unit (GPU). Furthermore, the explicit use of the terms “processor” or “controller” should not be interpreted as exclusively referring to hardware capable of running software, but may implicitly include, but are not limited to, digital signal processor (DSP) hardware, network processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), read-only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage. Other conventional and / or custom hardware may also be included.

[0073] A software module, or simply a module that is suggested to be software, may be represented herein as any combination of flowchart elements or other elements that indicate the execution of process steps and / or textual descriptions. Such modules may be executed by hardware, either explicitly or implicitly indicated.

[0074] Using these principles appropriately, the developers then examine several non-limiting examples to illustrate various implementations of the technology.

[0075] Referring to Figure 2, a schematic diagram of computing device 100 suitable for use in several non-restrictive implementations of this technology is shown.

[0076] Electronic devices The computing device 100 comprises various hardware components, including one or more single or multi-core processors collectively represented by a processor 110, a graphics processing unit (GPU) 111, a solid-state drive 120, random-access memory 130, a display interface 140, and an input / output interface 150.

[0077] Communication between the various components of the computing device 100 may be enabled by one or more internal buses and / or external buses 160 (e.g., PCI bus, Universal Serial Bus, IEEE 1394 "FireWire" bus, SCSI bus, Serial ATA bus, etc.), and these various hardware components are electronically coupled to the internal buses and / or external buses 160.

[0078] The input / output interface 150 may be coupled to the touchscreen 190 and / or one or more internal and / or external buses 160. The touchscreen 190 may be part of the display. In some implementations, the touchscreen 190 is the display. The touchscreen 190 is sometimes equated with the screen 190. In the implementation shown in Figure 2, the touchscreen 190 comprises touch hardware 194 (e.g., pressure-sensitive cells embedded in a layer of the display that enable detection of physical interaction between the user and the display) and a touch input / output controller 192 that enables communication with the display interface 140 and / or one or more internal and / or external buses 160. In some implementations, the input / output interface 150 may be coupled to a keyboard (not shown), mouse (not shown), or trackpad (not shown) that allows the user to interact with the computing device 100 in addition to, or instead of, the touchscreen 190.

[0079] In an implementation example of this technology, the solid-state drive 120 stores program instructions that are loaded into the random-access memory 130 and are suitable for execution by the processor 110 and / or GPU 111 to train a machine learning model to predict abdominal aortic aneurysm (AAA) growth based on its features in images. For example, the program instructions may be part of a library or application.

[0080] Computing device 100 may be implemented in the form of a server, desktop computer, laptop computer, tablet, smartphone, personal digital assistant, or any other device that can be configured to implement the Technology, as can be understood by those skilled in the art.

[0081] system Referring to Figure 3, a schematic diagram of communication system 200 (hereinafter referred to as System 200) is shown, and System 200 is suitable for implementing a non-limiting implementation example of the Technology. It should be clearly understood that the illustrated System 200 is merely an exemplary implementation example of the Technology. Therefore, the following description of System 200 is merely an exemplary example of the Technology. This description is not intended to limit the scope of the Technology or to describe its limitations. In some cases, useful examples of modifications to System 200 may also be described below. This is done solely to aid understanding and, again, is not intended to limit the scope of the Technology or to describe its limitations. These modifications are not an exhaustive list, and as those skilled in the art will understand, other modifications are likely possible. Furthermore, if this is not done (i.e., examples of modifications are not described), modifications are not possible and / or described should not be interpreted as the only way to implement that element of the Technology. As those skilled in the art will understand, this may not be the case. Furthermore, it should be understood that System 200 may provide simple implementations of the technology in some cases, and if so, they are presented in this form for the purpose of aiding understanding. As those skilled in the art will understand, various implementations of the technology can be more complex.

[0082] System 200 includes, in particular, a medical imaging device 210, a workstation computer 215, a server 230, and a database 235, all connected through a communication network 220 via their respective communication links 225 (which are not numbered separately).

[0083] In one or more implementation examples, at least a portion of the system 200 implements Picture Archiving and Communication System (PACS) technology.

[0084] The medical imaging device 210 is operated by a user (e.g., a physician or technician) to acquire medical images of a given patient's body. Acquisition parameters may be controlled via a workstation computer 215.

[0085] Medical imaging device The medical imaging device 210 is configured, in particular, to (i) collect images of a given subject's body according to acquisition parameters, and to (i) transmit the images to a workstation computer 215.

[0086] The medical imaging device 210 may include one of the following: a computed tomography (CT) scanner, a magnetic resonance imaging (MRI) scanner, a 3D ultrasound, etc.

[0087] In some implementations of this technology, the medical imaging device 210 may include multiple medical imaging devices, such as one or more of a computed tomography (CT) scanner, a magnetic resonance imaging (MRI) scanner, a 3D ultrasound scanner, etc.

[0088] The medical imaging device 210 may be configured with specific acquisition parameters to acquire images of the patient. In the context of this technology, images of the patient's body include the aorta and / or iliac arteries. Images of the patient's body may also include the thoracic region (e.g., ascending aorta, aortic arch, descending thoracic aorta) and / or the abdominal aortic region (e.g., superior abdominal aorta, inferior aorta, renal artery, lumbar artery), as well as the iliac arteries (e.g., common iliac artery, external iliac artery, internal iliac artery).

[0089] In one or more implementations, the medical imaging device 210 is configured to collect a set of images, each containing at least one image. In one or more implementations, the set of images may be static images. In one or more other implementations, the set of images may be dynamic images in the form of a multi-phase stack.

[0090] It will be understood, at least in some cases, that when acquiring images using a medical imaging device 210, a contrast agent, also known as a contrast agent, may be administered to the patient to increase the contrast of structures or fluids within the patient's body.

[0091] As a non-limiting example, in one or more implementations in which the medical imaging device 210 is implemented as a CT scanner, a CT protocol including a preoperative retrospective gated multi-detector CT (MDCT - 64-slice multi-slice CT scanner) using variable-dose radiation to capture the RR interval may be used.

[0092] As another non-limiting example, in one or more implementations where the medical imaging procedure includes an MRI scanner, the MR protocol may include steady-state T2-weighted fast-field echo (TE=2.6ms, TR=5.2ms, flip angle 110 degrees, fat suppression (SPIR), echo time 50ms, up to 25 cardiac phases, matrix 256×256, acquired voxel MPS (measurement, phase, and slice coding direction) 1.56 / 1.56 / 3.00mm, and reconstructed voxel MPS 0.78).

[0093] In one or more alternative implementations, the medical imaging device 210 may include, or be connected to, a workstation computer (not shown) for, among other things, control of acquisition parameters and image data transmission.

[0094] In one or more implementations, the workstation computer 215 may be provided together with the medical imaging device 210, for example, within its housing. In one or more other implementations, the workstation computer 215 may be implemented as a mobile device, such as a smartphone or tablet.

[0095] In one or more implementations, the medical imaging device 210 is part of a picture archiving and communication system (PACS) for storing and retrieving medical images, along with other electronic devices such as a server 230.

[0096] server Server 230 is configured, among other things, to train a set of machine learning (ML) models 250 to perform predictions of AAA increase in images.

[0097] The following sections of this specification will provide a more detailed explanation of how server 230 is configured to do so.

[0098] Server 230 can be implemented as a conventional computer server and may comprise some or all of the components of computing device 100 shown in Figure 2. In one example of one or more implementations of the Technology, Server 230 can be implemented as a Dell® PowerEdge® server running the Microsoft® Windows Server® operating system. Needless to say, Server 230 can be implemented with any other suitable hardware and / or software and / or firmware, or a combination thereof. In the illustrated non-limiting implementations of the Technology, Server 230 is a single server. In alternative non-limiting implementations of the Technology, the functions of Server 230 may be distributed and implemented through multiple servers (not shown).

[0099] Implementations of server 230 are well known to those skilled in the art. However, in short, server 230 comprises a communication interface (not shown) structured and configured to communicate with various entities via a communication network 220 (e.g., workstation computer 215 and other devices potentially coupled to network 220). Server 230 further comprises at least one computer processor (e.g., processor 110 or GPU 111 of computing device 100) operably connected to the communication interface and structured and configured to perform the various operations described herein.

[0100] In one or more implementations, the server 230 may be implemented as a computing device 100, or may comprise components of the computing device 100, such as a processor 110, a graphics processing unit (GPU) 111, a solid-state drive 120, random-access memory 130, a display interface 140, and an input / output interface 150.

[0101] It will be understood that server 230 may provide the output of one or more processing steps to another electronic device for display, verification, and / or troubleshooting. In a non-limiting example, server 230 may transmit images, calculated values, results, and machine learning parameters for display on a client device configured similarly to computing device 100, such as a smartphone or tablet.

[0102] Server 230 has access to a set of 250 machine learning (ML) models.

[0103] Machine Learning (ML) The set of ML models 250 includes, in particular, a set of feature extraction models 260, a set of classification ML models 270, and a set of segmentation ML models 280.

[0104] In the context of this technology, please understand that machine learning methods include deep learning methods.

[0105] From this point forward, ML models will simply be referred to as models.

[0106] Each of the 250 models in the set is parameterized, among other things, by model parameters and hyperparameters.

[0107] Model parameters are the model settings used to make predictions. These settings are either estimated from training data or learned; that is, the coefficients are selected during training based on an optimization strategy to produce predictions. Hyperparameters are model settings that determine the initial structure of the model and how the initial model is trained.

[0108] It should be understood that the number of model parameters to be initialized depends, in particular, on the type of model (i.e., classification or regression), the model architecture (e.g., DNN, SVM, random forest, ensemble tree, etc.), and the model hyperparameters (e.g., the number of layers, the type of layers, the number of neurons in the NN, the number of trees, etc.).

[0109] In one or more implementation examples, hyperparameters include one or more of the following: number of hidden layers and units, optimization algorithm, learning rate, momentum, activation function, mini-batch size, number of epochs, and dropout.

[0110] Feature extraction model The feature extraction model set 260, also known as the feature extractor set 260, is configured to extract features from images received from the medical imaging device 210.

[0111] The set of 260 feature extraction models may include one or more models.

[0112] The features extracted from the 260-set feature extraction model, also known as deep features, will be used by the predictive model (e.g., the classifier) ​​to perform their respective predictions.

[0113] In the context of this technology, the set of feature extraction models 260 is used to extract deep features that indicate AAA increase in the image. Deep features may be extracted, for example, from segmented tissue output by the segmentation model 280. In one or more alternative implementation examples, deep features may be extracted directly from the baseline image.

[0114] It will be understood that the features extracted by the feature extraction model set 260 can be represented in the form of feature vectors. The number of features in a feature vector is not limited. As an unrestricted example, the set of features may be a 1024 or 2048-dimensional feature vector.

[0115] The set of 260 feature extraction models may include one or more different types of feature extraction models for performing deep feature extraction from images. As a non-restrictive example, the initial feature extractor may be implemented as a ResNet model. The ResNet model may be pre-trained on ImageNet data. In such an implementation example, the model parameters of the portion trained before the common feature extractor may be obtained during model initialization.

[0116] One or more of the 260 feature extraction models may be based on one of the following: attention mechanisms, autoencoders, inception networks, DenseNet, generative adversarial networks (GANs), AlexNet, GoogleNet, VGG, etc.

[0117] Classification model The set of classification models, also known as the set of classifiers 270, is configured, among other things, to (i) receive features extracted from images of the aortic region and (ii) use the extracted features to make their respective predictions.

[0118] In the context of this technology, each prediction represents an AAA increase. In one or more implementations, each prediction may be either a non-significant AAA increase or a significant AAA increase.

[0119] The set of classification models 270 includes multiple classification models 270. The set of classification models 270 may be divided into subsets of classification models, each subset of classification models 270 may be configured to perform predictions based on different types of features, as described below.

[0120] As a non-restrictive example, classification model 270 may be implemented based on ensemble trees, support vector machines (SVMs), random forests, neural networks, etc.

[0121] In one or more alternative implementations, the set of models 250 may further include a set of regression models (not shown). The set of regression models may be configured to receive features extracted from images of the aortic region and, based on the extracted features, make predictions for each of the AAA increases. In non-limiting examples, the regression models may predict relative increases or numerical values ​​indicating AAA increases.

[0122] In the context of this technology, the model may be formed from multiple models as described above, and for example, an ML model may be referred to that comprises a feature extractor for extracting features from an image and a classifier for classifying the image using the extracted features.

[0123] Segmentation model The set of segmentation models 280 is configured to perform tissue segmentation in the aortic region. The set of segmentation models 280 includes one or more segmentation ML models.

[0124] In one or more implementations, the set of segmentation models 280 is configured to perform semantic segmentation of tissue, that is, the set of segmentation models 280 is configured to detect (i.e., define) all boundaries and distinguish (i.e., classify) different tissue types in images of the aorta acquired by the medical imaging device 210.

[0125] The segmentation model set 280 is configured to segment the outer wall of the aorta, the inner wall of the aorta, the lumen, and intraluminal thrombi (ILT). Therefore, the segmentation model 280 may classify each pixel in a medical image as one of the outer wall of the aorta, the inner wall of the aorta, the lumen, and intraluminal thrombi (ILT).

[0126] In one or more implementation examples, the set of segmentation models 280 includes two segmentation models, each configured to perform a specific segmentation task. In a non-limiting example, the set of segmentation models 280 may include a first segmentation model configured to perform foreground-background segmentation and a second segmentation model configured to perform multi-class segmentation for detecting lumen and calcification (if present). Such non-limiting examples of segmentation models are described in International Patent Application PCT / IB2022 / 051558, entitled “METHOD AND SYSTEM FOR SEGMENTING AND CHARACTERIZING AORTIC TISSUES,” filed by the same applicant on 22 February 2022, and are incorporated herein by reference.

[0127] In one or more implementation examples, the set of 280 segmentation models includes fully convolutional neural networks (FCNs).

[0128] The segmentation model 280 is trained to perform segmentation of the aortic region in an image. In one or more implementations, the segmentation model 280 may be trained to perform segmentation based on CT and / or MRI images.

[0129] In one or more implementation examples, the set of 280 segmentation models features a ResNet-based FCN architecture. Non-restrictive examples of ResNet include ResNet50 (50 layers), ResNet101 (101 layers), ResNet152 (152 layers), ResNet50V2 (50 layers with batch normalization), ResNet101V2 (101 layers with batch normalization), and ResNet152V2 (152 layers with batch normalization).

[0130] In one or more alternative implementation examples, the set of segmentation models 280 may be implemented based on one of the following: U-Net, V-Net, SegNet, AlexNet, GoogleNet, VGG, DeepLab, Masked R-CNN, GAN, ResNet, Visual Transformer, etc.

[0131] database The database 235 is configured to store, in particular (i) acquisition parameters and data relating to the medical imaging device 210, (ii) medical images or their references, including stacks of baseline images and stacks of follow-up images, (iii) labels associated with medical images and features extracted from medical images, (iv) model parameters and hyperparameters of a set of ML models 250, (v) datasets for training, testing, and validating the set of ML models 250, and (vi) data output by the set of ML models 250.

[0132] Database 235 is configured to store medical image stacks and videos. In one or more implementations, the database may store Digital Imaging and Communications in Medicine (DICOM) files, including, for example, DCM and DCM30 (DICOM3.0) file extensions. Additionally or alternatively, database 235 may store medical image files in Tag Image File Format (TIFF), Digital Storage and Retrieval (DSR) TIFF-based formats, and Data Exchange File Format (DEFF) TIFF-based formats.

[0133] In one or more implementations, database 235 may store ML file formats such as .tfrecords, .csv, .npy, and .petastorm, as well as file formats used to store models such as .pb and .pkl. Database 235 may also store, but is not limited to, well-known file formats such as image file formats (e.g., .png, .jpeg, .exif, .bmp, .tiff), video file formats (e.g., .mp4, .mkv, etc.), archive file formats (e.g., .zip, .gz, .tar, .bzip2), document file formats (e.g., .docx, .pdf, .txt), or web file formats (e.g., .html).

[0134] Communication network In some implementations of this technology, the communication network 220 is the Internet. In alternative, non-limiting implementations, the communication network 220 can be implemented as any suitable local area network (LAN), wide area network (WAN), private communication network, etc. It should be clearly understood that the implementation examples for the communication network 220 are for illustrative purposes only. How the communication link 225 (not separately numbered) between the workstation computer 215 and / or the server 230 and / or other electronic devices (not shown) and the communication network 220 is implemented depends, in particular, on how each of the medical imaging device 210, the workstation computer 215, and the server 230 is implemented.

[0135] The communication network 220 may be used to transmit data packets between the workstation computer 215, the server 230, and the database 235. For example, the communication network 220 may be used to transmit requests between the workstation computer 215 and the server 230.

[0136] Growth prediction training procedure Referring to Figure 4, a schematic diagram of the training dataset generation procedure 300 is shown, based on one or more non-exclusive implementations of this technology.

[0137] The training dataset generation procedure 300 is used to generate a training dataset and train one or more ML models 250 in order to predict AAA increase in images collected by a medical imaging device such as a medical imaging device 210.

[0138] In one or more alternative implementations of this technology, the training dataset generation procedure 300 may be adapted and used to predict the growth of other types of aneurysms within blood vessels, such as thoracic aneurysms.

[0139] The objective of the training dataset generation procedure 300 is to determine whether a patient exhibits significant or non-significant AAA increase by comparing baseline images received during a baseline imaging session with images received during a follow-up imaging session. The baseline images are then labeled accordingly, and different types of features (i.e., shape features, texture features, and deep features) are extracted from the baseline images. These baseline images may be segmented using a set of segmentation models 280. The contributions of the different types of features are then determined by training a set of classification models 270 to predict AAA increase in the images using the determined labels as ground truth.

[0140] The training dataset generation procedure 300 includes, among other things, an image acquisition procedure 320, a registration procedure 340, a segmentation procedure 350, a comparison procedure 360, a feature extraction procedure 500, a labeling procedure 370, a calcification measurement procedure 380, and a training and validation procedure 700.

[0141] Image collection procedure The image acquisition procedure 320 is configured to receive a set of baseline images and a set of follow-up images acquired by the medical imaging device 210 for each of the multiple patients.

[0142] The baseline image set includes at least one baseline image acquired during the first imaging session of a given patient, and the follow-up image set includes at least one follow-up image acquired during subsequent imaging sessions of the same given patient. In the context of this technology, each patient is diagnosed with an abdominal aortic aneurysm (AAA), and the purpose of the baseline and follow-up imaging sessions is to determine the growth of the AAA. It will be understood that in some implementations, two or more follow-up sessions may be considered. In at least some cases, each patient is administered a contrast agent before the baseline and follow-up imaging sessions to enhance the contrast of the baseline and follow-up image sets.

[0143] It will be understood that the baseline image set and the follow-up image set may be received at different times and / or from different devices (e.g., the medical imaging device 210 and / or the workstation computer 215) by the image acquisition procedure 320.

[0144] The baseline image set and the follow-up image set may each be in the form of an image stack, each containing multiple images (e.g., tens or hundreds of images).

[0145] An image stack, also called a slice, contains a set of sequential images that can be scrolled, and it will be understood that this is suitable for cross-sectional studies (e.g., CT / MRI) and time-resolved modalities. In a non-limiting example, an image stack may be provided in the DICOM file format.

[0146] In one or more implementations, the image stack may be in the form of a polyphasic stack, in which case each phase of the polyphasic stack may correspond to a time instance. In a non-limiting example, each phase in the stack may correspond to a moment in the cardiac cycle of a given patient.

[0147] In one or more alternative implementations where, for each patient, the baseline image set contains only one baseline image and the follow-up image set contains only one follow-up image, the total number of images (or patients) may need to be sufficient to ensure optimal training and prevent overfitting of the machine learning model.

[0148] In one or more alternative implementation examples, the baseline and follow-up images may be acquired by different types of medical imaging devices (e.g., CT and MRI) and may be aligned on the same reference frame during the alignment procedure 340.

[0149] The image acquisition procedure 320 outputs a set of baseline images and a set of follow-up images for each of the multiple patients, showing at least the aortic region.

[0150] It should be understood that instructions for a set of baseline images and instructions for a set of follow-up images of the same patient may be associated with each other as pairs; that is, a subset of baseline images may be associated with a corresponding subset of follow-up images.

[0151] Alignment procedure The alignment procedure 330 is configured to align the baseline image set with the follow-up image set and ensure correspondence between the images.

[0152] Alignment is performed to incorporate the modalities involved into a common reference frame (i.e., spatial alignment), so that the information they contain can be optimally integrated or compared.

[0153] As a non-limiting example, the alignment procedure 330 may be performed using software called Simpleware®ScanIP® (Synopsys Inc., Mountain View, California) to perform the alignment of the baseline image and the follow-up image.

[0154] The alignment procedure 330 outputs a aligned baseline image and a follow-up image. In one or more implementation examples, the alignment procedure 330 outputs a aligned baseline stack and a aligned follow-up stack.

[0155] Referring briefly to Figure 6, the baseline and follow-up slice 602 before alignment, and the baseline and follow-up slice 612 after alignment output by the alignment procedure 330 are shown.

[0156] Returning to Figure 4, step 300 includes the segmentation step 350.

[0157] Segmentation procedure The segmentation procedure 350 is configured, in particular, to (i) receive baseline and follow-up images, and (ii) segment the aortic tissue in the baseline and follow-up images.

[0158] The segmentation procedure 350 uses a set of segmentation models 280 that have been trained to segment aortic tissue in images acquired by an imaging device such as a medical imaging device 210.

[0159] The segmentation procedure 350 segments the tissue in the baseline and follow-up images using a set of segmentation models 280. The baseline and follow-up images may each be in the form of an image stack containing multiple slices.

[0160] The segmentation procedure 350 obtains a segmented aortic region for each pair of baseline images associated with follow-up images, including one or more of the aorta and iliac arteries. In one or more implementations, the segmented aortic region includes a region of interest (ROI) which includes the lumen, medial and lateral aortic wall, ILT (if present), and calcification (if present).

[0161] In one or more implementations, the segmentation procedure 350 is configured to extract segmented tissue for each pair of follow-up and baseline images, obtaining at least one image per segmented tissue. It will be understood that segmented tissue may be extracted by performing masking.

[0162] Referring briefly to Figure 6, we see pair 620, which includes the extracted baseline ROI 622 and the extracted follow-up ROI 626, output by the segmentation procedure 350.

[0163] Returning to Figure 4, the comparison procedure 360 ​​is performed following the segmentation procedure 350.

[0164] Comparison procedure The comparison procedure 360 ​​is configured to measure, for each patient, the difference between the segmented aortic region in the set of follow-up images and the segmented aortic region in the set of baseline images.

[0165] In one or more implementations, comparison procedure 360 ​​measures the difference in pixel count between a subset of baseline images and an associated subset of follow-up images. The subset of baseline images is a suitable subset of the baseline images, and the subset of follow-up images is a suitable subset of the follow-up images. Thus, the set of images may include subsets of multiple images. In one or more implementations, each subset contains one image. In one or more other implementations, each subset contains two or more images.

[0166] In one or more implementations, the comparison procedure 360 ​​measures the difference between the segmented aortic region in each slice of the baseline image stack and the segmented aortic region in the corresponding slice of the follow-up image stack.

[0167] In one or more alternative implementations, comparison procedure 360 ​​measures the difference in pixel count between the entire subset of baseline images and the entire subset of follow-up images.

[0168] In one or more implementations, the comparison procedure 360 ​​determines the number of non-zero pixels N of the extracted ROIs in the baseline. B and the number of non-zero pixels in the extracted ROI in the follow-up N F The following is measured. For each patient slice, the difference (D) between the baseline non-zero pixels and the follow-up non-zero pixels is calculated using equation (1). D=N F -N B (1)

[0169] In this way, comparison procedure 360 ​​calculates the difference in pixel count for each corresponding slice in the baseline image stack and the follow-up image stack.

[0170] The comparison procedure 360 ​​is configured to compare, for each patient, the difference between the segmented aortic region in the follow-up image and the segmented aortic region in the baseline image to at least one given threshold.

[0171] If the difference is greater than or equal to a given threshold, comparison procedure 360 ​​determines that the patient has a significant increase in AAA for a given pair (i.e., slice) of baseline and follow-up images. If the difference is less than a given threshold, comparison procedure 360 ​​determines that the patient does not have a significant increase in AAA for a given pair (i.e., slice) of baseline and follow-up images.

[0172] One or more alternative implementation examples may have two or more levels of AAA increase (threshold).

[0173] In one or more implementations, comparison procedure 360 ​​determines a threshold based on the third quartile (Q3) of the difference (D) of the measured pixels for all slices of all patients. If the difference is below the third quartile, comparison procedure 360 ​​determines that the patient has a non-significant AAA increase. If the difference is equal to or greater than the third quartile, comparison procedure 360 ​​determines that the patient has a significant AAA increase. It should be understood that the threshold may be determined using other techniques.

[0174] Comparison procedure 360 ​​determines whether the patient shows a significant or non-significant increase in AAA based on equations (2) and (3). Non-significant increase: D <Q3(2) Significant increase: D≧Q3(3)

[0175] The comparison procedure 360 ​​associates each slice of each patient with an indication of whether the patient has a non-significant or significant AAA increase. In non-limiting examples, this association may be stored in the database 235 or another storage medium. The comparison procedure 360 ​​may, for example, associate each baseline image of each patient with an indication of a non-significant or significant AAA increase.

[0176] As will be described in more detail below in this specification, the indications for non-significant or significant AAA increase will be used as labels for labeling the features extracted from each baseline image by the labeling procedure 370. The labels will be used as ground truth for training the machine learning model during the supervised learning procedure.

[0177] Procedure 300 is configured to perform a feature extraction procedure 500 to extract different types of features from the baseline image, which will be explained with reference to Figures 4 and 5.

[0178] Feature extraction procedure The feature extraction procedure 500 is used to extract different types of features from each of the baseline images so that a predictive model can be trained to predict AAA growth based on different types of features and its performance can be judged.

[0179] In one or more implementation examples, the feature extraction procedure 500 is performed by the server 230.

[0180] The feature extraction procedure 500 includes, in particular, a shape feature extraction procedure 520, a texture feature extraction procedure 540, and a deep feature extraction procedure 560.

[0181] The shape feature extraction procedure 520, the texture feature extraction procedure 540, and the deep feature extraction procedure 560 may be performed by the same electronic device or by different electronic devices, and it should be understood that these procedures may be performed in parallel or sequentially.

[0182] Procedure for extracting shape features The shape feature extraction procedure 520 is configured to extract shape features 522 from a baseline image. Shape features 522 are image features that indicate the shape of elements (e.g., objects) in the image.

[0183] Shape features may be invariant under the scaling, rotation, and translation of an object, and by their properties, it can be understood that they are either 2D or 3D depending on the object.

[0184] In one or more implementation examples, the shape feature extraction procedure 520 extracts shape features from segmented baseline images.

[0185] The shape feature extraction procedure 520 may extract shape features from at least one of the aortic wall, ILT, lumen, and calcifications that have been segmented in the baseline image by the segmentation procedure 350.

[0186] In one or more implementations, at least one of the following—aortic wall, ILT, lumen, and calcification (if present)—is extracted from each slice of the baseline image stack for each patient.

[0187] In one or more implementations, the shape feature extraction procedure 520 is configured to perform a gradient direction histogram (HOG) to extract shape features. Such shape features may be called HOG features.

[0188] Gradient histograms are commonly used for shape recognition in localized areas of an image, using intensity gradients and edge direction distributions. For this purpose, the image is divided into small cells, and a gradient histogram is calculated for each pixel within each cell. All histograms are concatenated to extract a final feature vector representing the shape in the image. A grayscale image of the lumen may be used. In one exemplary experiment, the image was not cropped to ensure that shape information was not lost. Instead, the image was resized to 400x400 pixels in an 8x8 cell size to accelerate computation during the exemplary experiment.

[0189] The shape feature extraction procedure 520 outputs segmented lumen shape features 522 for each baseline image.

[0190] In one or more implementations, the shape feature extraction procedure 520 associates the shape features 522 with indications of the patient and / or baseline images from which they were extracted, so that they can be retrieved in a database 235 or another storage medium during training.

[0191] Therefore, for a given patient, each subset of baseline images is associated with a shape feature 522, which includes shape features extracted from the lumen and / or shape features extracted from the ILT.

[0192] In one or more alternative implementations, it will be understood that shape features may be extracted for each segmented tissue in the baseline image.

[0193] Texture Feature Extraction Procedure The texture feature extraction procedure 540 is configured to extract texture features 542 from the baseline image.

[0194] In one or more implementations, the texture feature extraction procedure 540 receives segmented lumen 502 extracted from the baseline image.

[0195] The texture feature extraction procedure 540 extracts texture features 542 from the segmented lumen.

[0196] Image texture features quantify the perceived texture of an image and provide information about the spatial arrangement of color or intensity in the image or selected regions within the image. It should be understood that the extraction of image texture features may be performed using either structured or statistical methods.

[0197] In one or more implementations, the texture feature extraction procedure 540 is configured to perform texture analysis using a gray-level co-occurrence matrix (GLCM) and extract texture features. It should be understood that the image may be converted to grayscale before GLCM feature extraction.

[0198] Texture-based statistical methods are used to estimate the spatial distribution of gray levels in an image by specifying local features for each pixel and extracting statistics from their distribution. GLCM considers the spatial relationships between pixels in different orientations. The co-occurrence matrix is ​​extracted from the gray-level image and shows the frequency of occurrence of pixels with value i in the neighborhood of pixels with value j passing through horizontal, vertical, or diagonal directions specified by a given offset. Contrast, uniformity, correlation, and energy are top-level parameters that can be extracted from GLCM. Contrast is defined as the local gray-level variation in GLCM. Uniformity measures the non-zero uniformity of GLCM, meaning that lower uniformity indicates higher gray-level variation and therefore higher contrast. Correlation calculates the probability that a particular pair of pixels in the gray level occur simultaneously, and energy is a measure of texture uniformity, indicating local uniformity. Therefore, higher energy also indicates higher uniformity.

[0199] In one or more implementations, the texture feature extraction procedure 540 extracts GLCMs from each patch having a size of 3x3 pixels, where the pixel of interest is the pixel in the center of the patch.

[0200] In one or more implementations, to reduce computation time, all patches belonging to (and not segmented) the image background are ignored, ensuring that all features are extracted only from the lumen. Multiple GLCMs may be extracted based on the spatial relationships between pixels of interest and their neighbors at different orientations of 0, 45, 90, and 135 degrees.

[0201] In one or more implementation examples, for each GLCM, statistical properties are calculated using equations (4) to (6) as vectors of contrast, energy, and uniformity values. contrast:

number

number

number

[0202] The texture feature extraction procedure 540 outputs segmented lumen texture features 542 for each subset of the baseline image.

[0203] In one or more implementations, the shape feature extraction procedure 520 allows the texture features 542 to be retrieved during training by associating them with instructions for the baseline images from which they were extracted, in a database 235 or another storage medium.

[0204] Therefore, for a given patient, each baseline image is associated with a texture feature 542, which includes texture features extracted from the lumen.

[0205] In one or more alternative implementations, it will be understood that texture features may be extracted for each segmented tissue in the baseline image.

[0206] Deep Feature Extraction Procedure The deep feature extraction procedure 560 is configured to extract deep features 562 from a baseline image using one or more feature extraction models 260, also known as feature extractors 260.

[0207] In one or more implementations, 562 deep features are extracted for each subset of the baseline image set.

[0208] In one or more implementations, the deep feature extraction procedure 560 receives segmented lumen 502 and segmented ILT and wall 506 extracted from baseline images by the segmentation model 280.

[0209] In one or more implementation examples, the deep feature extraction procedure 560 uses the deep feature extractor 260 to extract deep features 562.

[0210] Convolutional neural networks (CNNs) are recognized as powerful feature extractors capable of providing all information in an image, from abstract-level information such as shape, boundaries, and edges to detailed texture information. CNNs are categorized as shallow networks, deep networks, and complex networks based on their architecture.

[0211] Non-restrictive examples of feature extractors 260 include autoencoders, inception networks, DenseNet, GANs, ResNet, VGG, GoogleNet, and AlexNet.

[0212] In one or more implementations, the feature extractor is implemented as VGG-19 to extract features from the lumen and ILT, with features extracted from the fully connected layer FC8, and each baseline slice represented by a 1 × 1000 size feature vector. Other implementations of the feature extractor are also possible and may fall within the scope of this technology.

[0213] The deep feature extraction procedure 560 extracts deep features from the segmented lumen 502.

[0214] The deep feature extraction procedure 560 extracts deep features from the segmented ILT and walls 504.

[0215] In one or more implementations, the deep feature extraction procedure 560 allows the deep features 562 to be retrieved during training by associating them with the tissue and baseline image references from which they were extracted in a database 235 or another storage medium.

[0216] Therefore, for a given patient, each baseline image is associated with a deep feature 562, which includes deep features generated based on the lumen 502, and / or deep features generated based on the ILT and wall 504.

[0217] Labeling procedure In one or more implementations, the labeling step 370 is configured to associate each type of feature obtained during the feature extraction step 500 (shape features, texture features, and deep features extracted from segmented ROIs) with a label indication based on the results of the comparison step 360.

[0218] Labeling procedure 370 assigns a label to each baseline image and / or each patient, where each label is either a non-significant AAA increase or a significant AAA increase. Labeling procedure 370 may then associate extracted features with their respective labels.

[0219] As a non-restrictive example, features may be stored in database 235 (or another storage medium) along with the indications (e.g., identifiers, pointers, addresses, and / or similar) of the baseline images from which they were extracted, and the labels associated with the baseline images (i.e., significant AAA increase or non-significant AAA increase). During training, it will be understood that features may be retrieved from storage and provided to the ML model, rather than being re-extracted from the same baseline images each time.

[0220] In one or more alternative implementations, the labeling procedure 370 is configured to generate a separate labeled training dataset for each type of extracted feature, each with its associated label.

[0221] Labeling procedure 350 outputs a label for each baseline image associated with a given patient, where each label is either a non-significant AAA increase or a significant AAA increase.

[0222] Calcification measurement procedure The calcification measurement procedure 380 is configured to measure the amount of calcification accumulation in images labeled as showing significant AAA increase and non-significant AAA increase for each patient.

[0223] The purpose of calcification measurement procedure 380 is to measure the amount of calcification accumulation in each patient and to clarify the relationship between aortic calcification and increased AAA.

[0224] In one or more implementations, calcification measurement procedure 380 uses segmented calcifications obtained from segmentation procedure 350 to determine the calcification accumulation. In one or more other implementations, calcification measurement procedure 380 uses a segmentation model from a set of segmentation models 280 to extract calcifications from baseline images.

[0225] The calcification measurement procedure 380 calculates the number of segmented calcification pixels in each slice for each patient. In some implementations, the calcification measurement procedure 380 may calculate the number of calcification pixels by using a threshold, for example, by calculating the third quartile of the number of calcification pixels for all slices in the baseline stack per patient.

[0226] Calcification measurement procedure 380 outputs the amount of calcification for each baseline for each patient.

[0227] Referring briefly to Figure 8, Chart 800 is shown, which displays the amount of calcification accumulation per pixel (y-axis) corresponding to patient number (x-axis), comparing calcification in slices labeled as significantly increased (right-hand bars) with calcification in slices labeled as not significantly increased (left-hand bars) for each patient. The results show a direct relationship between calcification and significant increase. For example, patient number 6 has the largest AAA increase, and the amount of calcification is higher in slices labeled as showing a significant increase (right-hand bars) compared to slices labeled as showing a not significantly increased AAA increase (left-hand bars). On the other hand, patients number 2 and 10 are examples of patients with the smallest AAA increase, and the amount of calcification is almost equal in slices showing a significant increase (right-hand bars) and slices showing a not significantly increased increase (left-hand bars).

[0228] Here, with reference to Figures 7 and 4, the training and verification procedure 700, using one or more non-exclusive implementations of this technology, will be described in more detail.

[0229] Training Procedure The training and verification procedure 700 is executed by the server 230. It should be understood that the training and verification procedure 700 may be distributed and executed by multiple computing devices, and some training subprocedures may be executed in parallel or sequentially.

[0230] The objective of the training and validation procedure 700 is to train a model to classify baseline images based on the features extracted and output by the feature extraction procedure 500, indicating significant or non-significant AAA increases. To achieve this objective, the classifier 270 is trained using the labels output by the labeling procedure 370 (i.e., indicating significant or non-significant increases) as the ground truth.

[0231] The training and validation procedure 700 extracts different types of features and the labels associated with each baseline image for each baseline image. It will be understood that the different types of features and associated labels may be stored in the database 235 or another storage medium.

[0232] In one or more implementations, the training and validation procedure 700 determines which features are the best predictors of AAA growth by performing separate training of the model based on each of the texture features, shape features, and deep features, as well as at least some combinations thereof (i.e., texture and shape features, and deep features).

[0233] In one or more implementations, the training and validation procedure 700 may be performed so that one or more classifiers 270 are trained to consider features of two or more baseline images in a subset of baseline images. In such implementations, it will be understood that one or more classifiers 270 may consider consecutive or discontinuous images (e.g., consecutive slices in an image stack or discontinuous slices in an image stack). Furthermore, one or more classifiers 270 may be further configured to consider, for multiple images in a subset of baseline images, features parallel to the sagittal plane (i.e., the z-axis or vertical growth direction) in addition to features parallel to the cross-section (i.e., the xy-axis or horizontal growth direction).

[0234] The training and validation procedure 700 performs separate training of the classification model 270 on features extracted only from the lumen, features extracted only from the segmented ILT, and features extracted from a combination of the segmented lumen and the segmented ILT.

[0235] The training and verification procedure 700 includes, in particular, the first training and verification procedure 710, the second training and verification procedure 720, the third training and verification procedure 730, the fourth training and verification procedure 740, the fifth training and verification procedure 750, and the sixth training and verification procedure 760. It will be understood that procedures 710, 720, 730, 740, 750, and 760 may be performed at different points in time, sequentially, or in parallel.

[0236] In one or more implementations, the training and validation procedure 700 initializes a set of classification models 270. In one or more other implementations, the training and validation procedure 700 receives a set of classification models 270 that have already been initialized and / or pre-trained.

[0237] In one or more implementation examples, the set of 270 classification models is implemented as an ensemble tree classifier.

[0238] The training and validation procedure 700 performs training on each of the 270 sets of classification models.

[0239] It will be understood that training and validation procedure 700 can compare and judge the performance of models with different types of features by using the same initial model in the set of classification models 270 when performing training and validation procedures 710, 720, 730, 740, 750, and 760. Subsequently, training and validation procedure 700 may be repeated for different types of models with different types of features, thereby determining the type of features (or combination of features) that best demonstrates AAA growth.

[0240] During training, each of the 270 classification models performs classification of each baseline image based on the provided features. Then, a loss function is used to calculate the loss based on the predictions and the labels associated with the baseline images, and the parameters of the classification model 270 are updated based on the calculated loss. As is well known in the art, this procedure is repeated iteratively until convergence and / or until a stopping criterion is reached. In one or more implementation examples, the training and validation procedure 700 may be stopped when one or more of the following are reached: a desired performance threshold (e.g., the accuracy of the classification task at which overfitting is minimized), computational budget, maximum training duration, lack of improvement in performance, system failure, etc.

[0241] Next, the training and validation procedure 700 outputs a set of 270 pre-trained classification models, also called the pre-trained classifiers 270, each classifier being trained on different features.

[0242] Shape features The first training and validation procedure 710 trains at least one given classification model 270 to predict AAA increase in an image based on luminal shape features 702.

[0243] The classification model 270 uses luminal shape features 702 to classify each baseline image as either showing a significant AAA increase or a non-significant AAA increase.

[0244] In one or more implementations, the first training and validation procedure 710 trains a classification model 270 based on HOG features extracted from the lumen in each slice to predict AAA increase. HOG features represent the lumen shape.

[0245] Next, validation is performed using a portion of the dataset to fine-tune the model parameters of the given classification model 270.

[0246] When the termination condition is reached, the first training and validation procedure 710 stops and outputs a given trained classification model 270 trained on the lumen shape features 702.

[0247] Texture features The second training and validation procedure 720 trains at least one given classification model 270 to classify images based on luminal texture features 704.

[0248] The classification model 270 uses luminal texture features 704 to classify each image as either showing a significant or non-significant increase in AAA.

[0249] In one or more implementations, the second training and validation step 720 trains an ensemble tree classifier based on GLCM features extracted from the lumen in each slice, along with corresponding augmentation labels, to predict AAA augmentation by considering only the heterogeneity of lumen contrast.

[0250] Next, validation is performed using a portion of the dataset to fine-tune the model parameters of the given classification model 270.

[0251] When the termination condition is reached, the second training and validation procedure 720 stops and outputs a given trained classification model 270 trained on the lumen texture features 704.

[0252] Shape and texture features The third training and validation procedure 730 trains at least one given classification model 270 to classify images based on lumen shape features 702 and lumen texture features 704.

[0253] In one or more implementations, a third training and validation step 730 trains a set of classification models 270 to predict AAA augmentation by classifying images based on a combination of GLCM and HOG features extracted from the lumen in each slice, along with corresponding augmentation labels, thereby taking into account both luminal shape and luminal contrast heterogeneity.

[0254] When the termination condition is reached, the third training and validation procedure 730 stops and outputs a given classification model 270 trained on the lumen shape features 702 and lumen texture features 704.

[0255] Next, validation is performed using a portion of the dataset to fine-tune the model parameters of the given classification model 270.

[0256] The third training and validation procedure 730 outputs a given pre-trained classification model 270 that has been trained to perform AAA augmentation predictions based on lumen shape features 702 and lumen texture features 704.

[0257] Deep Lumen Features The fourth training and validation procedure 740 trains a set of classification models 270 to classify images based on the deep lumen features 712 extracted by the trained feature extractor 260.

[0258] In one or more implementations, the fourth training and validation procedure 740 trains at least one given classification model 270 to classify images based on luminal deep features 712 extracted in each baseline slice along with corresponding augmentation labels, and to predict AAA augmentation by taking all luminal features into consideration.

[0259] Next, validation is performed using a portion of the dataset to fine-tune the model parameters of the given classification model 270.

[0260] When the termination condition is reached, the fourth training and validation procedure 740 stops and outputs a given trained classification model 270 trained on the luminal deep features 712.

[0261] ILT and wall depth features The fifth training and validation procedure 750 trains at least one given classification model 270 to classify images based on ILT and wall depth features 714 extracted by the trained feature extractor 260.

[0262] In one or more implementations, the fifth training and validation step 750 trains at least one classification model 270 to classify images based on ILT and wall deep features 714 extracted in each slice along with corresponding augmentation labels, and to predict AAA augmentation taking into account all attributes of the ILT and walls.

[0263] Next, validation is performed using a portion of the dataset to fine-tune the model parameters of the given classification model 270.

[0264] Upon reaching the termination condition, the fifth training and validation procedure 750 stops and outputs at least one trained classification model 270 trained based on the ILT and wall deep features 714.

[0265] Luminous deep features and ILT and wall deep features The sixth training and validation procedure 760 trains at least one given classification model 270 to classify images based on the deep lumen features 712 and ILT and deep wall features 714 extracted by the trained feature extractor 260.

[0266] When the end condition is reached, the sixth training and verification procedure 760 stops and outputs at least one trained classification model 270 trained based on the lumen deep feature amount 712 and the ILT and wall deep feature amount 714.

[0267] Therefore, the training and verification procedure 700 outputs a plurality of trained classification models 270, and each trained classification model 270 is trained for each one of the lumen shape feature amount 702, the lumen texture feature amount 704, the lumen shape feature amount 702 and the lumen texture feature amount 704, the lumen deep feature amount 712, the ILT and the wall deep feature amount 714, the lumen deep feature amount 712 and the ILT and the wall deep feature amount 714.

[0268] It should be understood that features from other segmented tissues may be considered, and additional classification models 270 may be trained for features from other segmented tissues not described above.

[0269] In this way, the performance of each trained classification model 270 in predicting AAA enlargement may be compared and judged.

[0270] Experimental results In one or more implementations, a set 270 of classification models was implemented as an ensemble tree classifier and trained using the RUSBoost method that combines random undersampling and boosting to handle data with unbalanced class samples.

[0271] Training and testing were performed in different steps to evaluate the contribution of each factor in AAA enlargement prediction. The training dataset included images of 10 patients with 100 slices each (a total of 1000 slices). In each step of training, the set 270 of classification models was trained on 80% of the slices and tested on the remaining 20% of the slices.

[0272] Referring briefly to Figure 9, a plot of test classification error (y-axis) corresponding to the number of trees (x-axis) is shown. By evaluating the classifier's performance with 1000 trees, the number of trees was set to 684 (Figure 9), and the learning rate was set to 0.01.

[0273] The contribution of each feature set to the growth prediction was determined by calculating the confusion matrix at each step and evaluating the classifier performance. The confusion matrix was calculated using the following method, by defining the positive class as a non-significant growth and the negative class as a significant growth. [Table 1]

[0274] Here, TP represents CT slices correctly classified as non-significant increases, FP determines CT slices incorrectly classified as non-significant increases, TN represents CT slices correctly classified as significant increases, and FN determines CT slices incorrectly classified as significant increases.

[0275] Confusion matrices were used to measure accuracy, sensitivity, and specificity at each step of the process. [Table 2]

[0276] The developers noted that these results indicate that lumen and ILT features are powerful features in predicting significant increases. Lumen shape shows a higher contribution than lumen contrast in predicting increases. The combination of lumen shape and lumen contrast is more powerful in predicting significant increases than lumen shape alone or lumen contrast alone. The feature extractor extracted features approximately 10 times faster than handcrafted features. These results indicate that the balance between predictions makes automated features more descriptive than handcrafted features.

[0277] Therefore, it is understood that luminal shape features extracted from baseline images of patients diagnosed with AAA may indicate increased AAA in patients when classified by a trained classifier using one or more implementations of this technology. Luminal shape features may also be combined with luminal contrast features to indicate increased AAA when classified by a trained classifier using this technology. Deep features of the lumen and / or ILT output by the feature extractor may also be used to perform predictions.

[0278] Furthermore, the amount of calcification accumulation in the baseline image may be calculated to provide further indication of AAA increase.

[0279] Method explanation Figure 10 is a flowchart of Method 1000 for training at least one model to predict abdominal aortic aneurysm (AAA) growth in an image, Method 1000 is performed according to one or more non-limiting implementations of the present technique.

[0280] In one or more alternative implementations of this technology, Method 1000 may be adapted and used to train a model to predict the growth of other types of aneurysms within blood vessels, such as thoracic aneurysms.

[0281] In one or more implementations, the server 230 includes at least one processor, such as a processor 110 and / or a GPU 111, operably connected to a non-temporary computer-readable storage medium, such as a solid-state drive 120 and / or random-access memory 130 that stores computer-readable instructions. At least one processor is configured or operable to execute method 1000 when it executes a computer-readable instruction.

[0282] Method 1000 begins with processing step 1002.

[0283] According to processing step 1002, at least one processor receives, for each patient among a plurality of patients, a set of baseline images and a set of follow-up images collected by the medical imaging device 210.

[0284] In one or more implementations, the set of baseline images includes a baseline image stack collected by the medical imaging device 210, and the set of follow-up images includes a follow-up image stack collected by the medical imaging device 210. Each of the baseline image stack and the follow-up image stack may be a multiphase stack.

[0285] Each patient has been previously diagnosed with AAA. It will be appreciated that in alternative implementations, the patient may be diagnosed with another type of aneurysm, such as a thoracic aneurysm.

[0286] It will be appreciated that the set of baseline images and the set of follow-up images may be received at different times. The set of baseline images is collected during a first or baseline imaging session, and the set of follow-up images is collected during a second or follow-up imaging session.

[0287] According to processing step 1004, at least one processor compares, for each patient among a plurality of patients, the set of baseline images and the set of follow-up images, and determines the respective differences in the aorta region for each subset of the follow-up images associated with the baseline images.

[0288] In one or more implementations, to perform the comparison, prior to processing step 1004, the processor performs alignment of the set of baseline images and the set of follow-up images to compare the same imaged anatomical region / slice, e.g., the same aorta region, in a baseline image and one follow-up image.

[0289] In one or more implementations, for each patient, at least one processor performs a comparison between a set of baseline images and a set of follow-up images by comparing each subset of baseline images with the associated subset of follow-up images. It should be understood that a subset of baseline images may contain at least one image, and a subset of follow-up images may contain at least one corresponding follow-up image.

[0290] In one or more implementations, the comparison between a set of images and a set of baseline images may be performed image by image (i.e., only one image per subset of baseline and follow-up images). In one or more alternative implementations, the comparison between a set of images may be performed on multiple images in a subset of baseline images and a subset of follow-up images, and may include a comparison of regions parallel to the sagittal plane (y-direction or perpendicular direction) to determine augmentation in 3D.

[0291] In one or more implementations, the processor obtains a set of differential (e.g., pixel) measurements in the aortic region for each patient.

[0292] According to processing step 1006, in response to each difference in the aortic region exceeding a threshold, at least one processor labels each subset of baseline images with a significant AAA enhancement label for each patient.

[0293] In one or more implementations, each subset of baseline images contains a single baseline image, and at least one processor labels each baseline image of the aortic region exceeding the threshold with a significant AAA enhancement label. In one or more other implementations, each subset of baseline images contains multiple baseline images, and the processor may label each subset of baseline images with a significant AAA enhancement label.

[0294] In one or more implementations, the processor calculates the difference in the number of pixels in the aortic region between the baseline image and the corresponding subsets of the follow-up image.

[0295] According to processing step 1008, in response to each difference in the aortic region falling below a threshold, at least one processor labels each baseline image with a non-significant AAA enhancement label for each of the multiple patients.

[0296] In one or more implementations, each subset of baseline images contains a single baseline image, and the processor labels each baseline image of the aortic region exceeding the threshold with a non-significant AAA augmentation label.

[0297] In one or more other implementations, each subset of baseline images may contain multiple baseline images, and the processor may label each subset of baseline images with a non-significant AAA augmentation label.

[0298] In one or more implementations, at least one processor determines a threshold based on the third quartile (Q3) of the pixel difference (D) measured for all slices of all patients. If the difference is below the third quartile, the processor determines that the patient has a non-significant increase in baseline and follow-up image subsets. If the difference is greater than or equal to the third quartile, comparison procedure 360 ​​determines that the patient has a significant increase in baseline and follow-up image subsets.

[0299] According to processing step 1010, at least one processor trains at least one ML model to classify baseline images based on their features by using each label as a target. Processing step 1010 includes processing steps 1012 to 1016.

[0300] In one or more implementations, at least one ML model is a set of 270 classifiers. In one or more implementations, each of the 270 classifiers may be implemented as an ensemble tree.

[0301] In one or more implementations, the set of classifiers 270 may be configured to consider and correlate features between two or more consecutive or discontinuous baseline images. In such implementations, the set of classifiers 270 may be further configured to consider augmentation in 3D for subsets of images containing multiple images (for example, by considering the z direction in addition to the x and y directions).

[0302] According to processing step 1012, the processor extracts each set of features from the aortic region for each subset of baseline images of each patient.

[0303] In one or more implementations, each set of features includes luminal shape features. The shape features may also include HOG features.

[0304] In one or more other implementations, each set of features includes at least texture features that show lumen contrast. The texture features may also include GLCM features.

[0305] In one or more implementations, the processor uses a feature extractor 260 to extract deep features as part of each set of features. Deep features may be extracted from segmented lumen and / or segmented wall and ILT.

[0306] According to processing step 1014, the processor uses at least one ML model to classify each subset of baseline images based on a set of features as either showing a significant AAA increase or a non-significant AAA increase, and obtains predictions for each AAA increase.

[0307] According to processing step 1016, the processor updates the parameters of at least one ML model based on each prediction and each label. In one or more implementation examples, at least one ML model is a set of classifiers 270.

[0308] The loss function is used to calculate the loss based on the respective labels associated with the predictions and baseline images, and the parameters of the classification model 270 are updated based on the calculated loss.

[0309] Processing steps 1012-1016 are repeated until a termination condition is reached. The termination condition may include a desired performance threshold (e.g., accuracy of the classification task), computation budget, maximum training duration, lack of performance improvement, system failure, etc.

[0310] According to processing step 1018, the processor outputs a trained ML model. The processor outputs at least one trained classifier 270.

[0311] It will be understood that at least one trained classifier 270 is trained to classify baseline images of patients. At least one trained classifier 270 has learned one or more functions that correlate features extracted from the aortic region with the presence or absence of AAA augmentation.

[0312] It should be understood that in some implementations, at least one trained classifier 270 is trained according to the labels of a subset of baseline images, each containing one label per baseline image, while in other implementations, at least one trained classifier 270 may learn to consider a subset of baseline images with multiple baseline images, and / or the relationships between baseline images in a set of baseline images, in order to determine augmentation.

[0313] In one or more other implementations, at least one trained classifier 270 may be further configured to consider scaling in 3D.

[0314] Method 1000 then terminates.

[0315] It should be understood that Method 1000 may be run multiple times using different classifiers for different types of features (including combinations thereof), and the performance of the classifiers may be compared.

[0316] Figure 11 shows a flowchart of Method 1100 for predicting AAA enlargement in images of patients diagnosed with AAA using at least one model, Method 1100 is performed according to one or more non-limiting implementations of the present technique.

[0317] Method 1100 may be performed after Method 1000, i.e., after training the classifier 270 and selecting the classifier that has the best performance. In one or more alternative implementations of the technique, Method 1100 may be adapted and used to train a model to predict the growth of other types of aneurysms in blood vessels, such as thoracic aneurysms.

[0318] In one or more implementations, the server 230 comprises at least one processor, such as a processor 110 and / or a GPU 111, operably connected to a non-temporary computer-readable storage medium, such as a solid-state drive 120 and / or random-access memory 130 for storing computer-readable instructions. The processor is configured or operable to execute method 1100 when it executes a computer-readable instruction.

[0319] Method 1100 begins with processing step 1102.

[0320] According to processing step 1102, at least one processor receives a set of images of a given patient's body, including the aorta, the set of images including at least one image, and the set of images is acquired using a medical imaging device 210.

[0321] In one or more implementations, the set of images is in the form of a stack.

[0322] According to processing step 1102, at least one processor segments each of the set of images using at least one trained segmentation model 280 to extract the aortic region containing at least the lumen of a given patient.

[0323] According to processing step 1104, at least one processor extracts a set of features from the lumen that includes a lumen shape feature that indicates the shape of the lumen.

[0324] In one or more implementations, the set of features includes a set of lumen texture features that show at least the contrast of the lumen of a given patient, and the classification is further based on the set of lumen texture features.

[0325] In one or more implementation examples, the set of luminal shape features includes HOG features.

[0326] In one or more implementation examples, the set of luminal texture features includes GLCM features.

[0327] In one or more implementations, the extraction of a set of features from the lumen is performed by a feature extraction model 260, where the set of features corresponds to deep features.

[0328] In one or more implementation examples, the set of features may include ILT and wall features.

[0329] According to processing step 1106, the processor uses the trained classifier 270 to classify each of the set of images as either showing AAA increase or not showing AAA increase, based on at least a set of features, and obtains a set of classified images for a given patient.

[0330] According to processing step 1108, the processor predicts whether the patient will have AAA enlargement based on at least a classified set of images.

[0331] In one or more implementations, the processor uses at least one trained segmentation model to segment calcification in each aortic region of a set of images, calculates the amount of calcification in each of the classified sets of images, and determines whether the amount of calcification exceeds a threshold. In response to the amount of calcification exceeding the threshold, the processor outputs the amount of calcification as a further indicator of AAA increase for a given patient.

[0332] According to processing step 1110, the processor outputs a prediction.

[0333] It should be understood that the predictions provided by this method and system are merely indications of AAA increase based on image features in the patient's images and may be combined with other methods for determining AAA increase in patients already diagnosed with AAA. As a non-limiting example, the predictions may be used in combination with other features / types of features for determining AAA increase.

[0334] In some cases, examples of modifications to the Technology may be included, which may be considered useful. This is done solely for the purpose of aiding understanding and, again, not to limit the scope of the Technology or to describe its limitations. These modifications are not an exhaustive list, and a person skilled in the art may still make other modifications while remaining within the scope of the Technology. Furthermore, where no examples of modifications are provided, modifications should not be construed as impossible and / or as the described modifications being the only way to implement that element of the Technology.

[0335] Modifications and improvements to the implementation examples of this technology described above may be apparent to those skilled in the art. The above description is illustrative and not limiting.

Claims

1. A method for training at least one classifier to predict the enlargement of abdominal aortic aneurysms (AAA) in images acquired by a medical imaging device, wherein the method is performed by at least one processor, For each of the multiple patients, A step of receiving a set of baseline images and a set of follow-up images collected by the medical imaging device in different imaging sessions, wherein each patient has been diagnosed with AAA. The steps include comparing the set of baseline images with the set of follow-up images to determine the difference in the aortic region for each subset of the baseline images and the associated subset of the follow-up images, In response to each of the above differences in the aortic region exceeding a threshold, The steps include assigning a significant AAA enhancement label to each subset of the baseline images, In response to the fact that the respective differences in the aortic region of each subset of the baseline images and the associated subset of the follow-up images fall below a threshold, The steps include assigning non-significant AAA enhancement labels to each subset of the baseline images, For each subset of the baseline image, the step of extracting each set of features from the aortic region, A step of training at least one classifier to classify the set of baseline images based on each set of features by using each of the aforementioned labels as a target, wherein for each baseline image, Using the at least one classifier, the steps include classifying each subset of the baseline image based on the set of features as either showing a significant AAA increase or a non-significant AAA increase, and obtaining predictions for each AAA increase; A step of updating at least one parameter of the at least one classifier based on each of the aforementioned predictions and each of the aforementioned labels. The training steps include, The steps include: outputting a trained classifier with updated parameters and Methods that include...

2. The method according to claim 1, wherein each set of the features includes shape features extracted from the lumen in the baseline image.

3. The method according to claim 2, wherein the shape feature includes a gradient direction histogram (HOG) feature.

4. The method according to any one of claims 1 to 3, wherein each set of the features includes a texture feature representing contrast extracted from the lumen in the baseline image.

5. The method according to claim 4, wherein the texture feature includes a gray-level co-occurrence matrix (GLCM) feature.

6. The method according to claim 1, wherein the step of extracting each set of features for each of the baseline images includes the step of extracting deep features from the lumen in the images collected by the medical imaging device using at least one feature extractor.

7. The method according to claim 6, wherein the at least one feature extractor is configured to extract deep features from the lumen, the blood vessel wall, and the intraluminal thrombus (ILT).

8. The method according to claim 7, wherein at least the feature extractor includes a convolutional neural network (CNN).

9. The method according to any one of claims 1 to 8, wherein the at least one classifier includes an ensemble tree.

10. The method according to any one of claims 1 to 9, wherein each subset of the baseline images comprises a plurality of the baseline images, and each associated subset of the follow-up images comprises a plurality of associated follow-up images.

11. The method according to claim 10, wherein each of the differences in the aortic region includes the differences in the aortic region parallel to the transverse plane and the aortic region parallel to the sagittal plane.

12. A method for predicting AAA growth based on at least one image of a given patient previously diagnosed with abdominal aortic aneurysm (AAA), wherein the method is performed by at least one processor, the processor having access to a trained classifier trained to classify the patient's image as either indicating AAA growth or not indicating AAA growth, and the method is The step of receiving a set of images of the body of a given patient, including the aorta, wherein the set of images includes at least one image, and the set of images is collected by a medical imaging device. The steps include: segmenting each of the set of images using at least one trained segmentation model to extract the aortic region including the lumen of the given patient; The steps include extracting a set of feature quantities from the lumen that includes a lumen shape feature quantity indicating the shape of the lumen, The steps include using the pre-trained classifier to classify each subset of the set of images as either showing AAA increase or not showing AAA increase, based on at least the set of features, and obtaining a set of classified images for the given patient. A step of predicting whether the patient will have increased AAA levels based on at least the set of classified images, The steps of outputting the prediction and Methods that include...

13. The steps include segmenting the calcification in each of the aortic regions of the set of images using the at least one trained segmentation model, The steps include calculating the amount of calcification in each of the aforementioned classified sets of images, The steps include determining whether the amount of calcification exceeds a threshold, In response to the amount of calcification exceeding the threshold, The steps include outputting the amount of calcification as a further indicator of AAA increase for the given patient, and The method according to claim 12, further comprising:

14. The set of features includes a set of lumen texture features that show the contrast of the lumen of the given patient, The method according to claim 13 or 14, wherein the classification is further based on the set of lumen texture features.

15. The method according to any one of claims 13 to 15, wherein the set of lumen shape features includes gradient direction histogram (HOG) features.

16. The method according to any one of claims 13 to 16, wherein the set of luminal texture features includes gray-level co-occurrence matrix (GLCM) features.

17. The method according to any one of claims 13 to 16, wherein the step of extracting the set of features from the lumen is performed by a feature extraction machine learning (ML) model, and the set of features corresponds to deep features.

18. The method according to any one of claims 12 to 17, wherein the trained classifier includes an ensemble tree.

19. A system for training at least one classifier to predict the enlargement of abdominal aortic aneurysms (AAA) in images acquired by a medical imaging device, A non-temporary computer-readable medium for storing instructions, At least one processor operably connected to the aforementioned non-temporary computer-readable medium and Equipped with, When the at least one processor executes the instruction, For each of the multiple patients Receiving a set of baseline images and a set of follow-up images collected by the medical imaging device in different imaging sessions, wherein each patient has been diagnosed with AAA. The baseline image set and the follow-up image set are compared, and the difference in the aortic region is calculated for each subset of the baseline images and the associated subset of the follow-up images. In response to each of the above differences in the aortic region exceeding a threshold, Each subset of the baseline images is assigned a significant AAA enhancement label, In response to the fact that the respective differences in the aortic region of each subset of the baseline images and the associated subset of the follow-up images fall below a threshold, Each of the subsets of the baseline images is assigned a non-significant AAA enhancement label, For each subset of the baseline image, extract each set of features from the aortic region, Training at least one classifier to classify the set of baseline images based on each set of features by using each of the aforementioned labels as a target, wherein for each baseline image, Using the at least one classifier, the respective subsets of the baseline image are classified based on the set of features as either showing a significant AAA increase or a non-significant AAA increase, and predictions for each AAA increase are obtained. Updating at least one parameter of the at least one classifier based on each of the aforementioned predictions and each of the aforementioned labels. This includes training, Output a trained classifier that includes the updated parameters. A system configured to perform the following actions.

20. The system according to claim 19, wherein each set of the features includes shape features extracted from the lumen in the baseline image.

21. The system according to claim 20, wherein the shape feature includes a gradient direction histogram (HOG) feature.

22. The system according to any one of claims 19 to 21, wherein each set of the features includes a texture feature representing contrast extracted from the lumen in the baseline image.

23. The system according to claim 22, wherein the texture features include gray-level co-occurrence matrix (GLCM) features.

24. The system according to claim 19, wherein, for each of the baseline images, extracting each set of features includes using at least one feature extractor to extract deep features from the lumen in the images collected by the medical imaging device.

25. The system according to claim 24, wherein the at least one feature extractor is configured to extract deep features from the lumen, the blood vessel wall, and the intraluminal thrombus (ILT).

26. The system according to claim 25, wherein at least the feature extractor includes a convolutional neural network (CNN).

27. The system according to any one of claims 19 to 26, wherein the at least one classifier includes an ensemble tree.

28. The system according to any one of claims 19 to 27, wherein each subset of the baseline images comprises a plurality of the baseline images, and each associated subset of the follow-up images comprises a plurality of associated follow-up images.

29. The system according to claim 28, wherein each of the differences in the aortic region includes the differences in the aortic region parallel to the transverse plane and the aortic region parallel to the sagittal plane.

30. A system for predicting AAA growth based on at least one image of a given patient previously diagnosed with abdominal aortic aneurysm (AAA), A non-temporary computer-readable medium for storing instructions, At least one processor operably connected to the aforementioned non-temporary computer-readable medium and Equipped with, The at least one processor has access to a trained classifier trained to classify patient images as either showing AAA increase or not showing AAA increase. When the at least one processor executes the instruction, Receiving a set of images of the body of a given patient, including the aorta, wherein the set of images includes at least one image, and the set of images is collected using a medical imaging device. Using at least one trained segmentation model, segment each of the set of images and extract the aortic region including the lumen of the given patient, Extracting a set of feature quantities from the lumen, including a lumen shape feature quantity that indicates the shape of the lumen, Using the pre-trained classifier, classify each subset of the set of images as either showing AAA increase or not showing AAA increase, based on at least the set of features, and obtain a set of classified images for the given patient. To predict whether the patient will have an increase in AAA levels based on at least the set of classified images, Outputting the aforementioned prediction A system configured to perform the following actions.

31. The aforementioned at least one processor is Segmenting the calcification in each of the aortic regions of the set of images using the at least one trained segmentation model, Calculate the amount of calcification in each of the aforementioned classified sets of images, To determine whether the amount of calcification exceeds a threshold, In response to the amount of calcification exceeding the threshold, The amount of calcification is output as a further indicator of AAA increase for the given patient. The system according to claim 30, further configured to perform the following:

32. The set of features includes a set of lumen texture features that show the contrast of the lumen of the given patient, The classification is further based on the set of lumen texture features according to claim 31 or 32.

33. The system according to any one of claims 31 to 33, wherein the set of lumen shape features includes gradient direction histogram (HOG) features.

34. The system according to any one of claims 31 to 33, wherein the set of luminal texture features includes gray-level co-occurrence matrix (GLCM) features.

35. The system according to any one of claims 31 to 34, wherein the extraction of the set of features from the lumen is performed by a feature extraction machine learning (ML) model, and the set of features corresponds to deep features.

36. The system according to any one of claims 31 to 35, wherein the trained classifier includes an ensemble tree.

37. A method for training at least one classifier to predict the growth of an aneurysm in images acquired by a medical imaging device, wherein the method is performed by at least one processor, For each of the multiple patients, A step of receiving a set of baseline images and a set of follow-up images collected by the medical imaging device in different imaging sessions, wherein each patient has been diagnosed with the aneurysm. The steps include comparing the baseline image set with the follow-up image set, and determining the differences in the vascular region for each subset of the baseline images and the associated subset of the follow-up images, In response to each of the aforementioned differences in the vascular region exceeding a threshold, The steps include labeling each subset of the baseline images with a significant aneurysm enlargement label, In response to the fact that the respective differences in the vascular region of each subset of the baseline image and the associated subset of the follow-up image fall below a threshold, The steps include labeling each subset of the baseline images with a non-significant aneurysm enlargement label, For each subset of the baseline image, the steps include extracting each set of features from the aortic region, A step of training at least one classifier to classify the set of baseline images based on each set of features by using each of the aforementioned labels as a target, wherein for each baseline image, Using at least one of the classifiers, the steps include classifying each subset of the baseline images based on the set of features as either indicating significant aneurysm enlargement or non-significant aneurysm enlargement, and obtaining predictions for each aneurysm enlargement; A step of updating at least one parameter of the at least one classifier based on each of the aforementioned predictions and each of the aforementioned labels. The training steps include, The steps include: outputting a trained classifier with updated parameters and Methods that include...

38. A method for training at least one classifier to predict the growth of an aneurysm in images acquired by a medical imaging device, wherein the method is performed by at least one processor, For each of the multiple patients, A step of receiving a set of baseline images and a set of follow-up images collected by the medical imaging device in different imaging sessions, wherein each patient has been diagnosed with the aneurysm. The steps include comparing the baseline image set with the follow-up image set, and determining the differences in the vascular region for each subset of the baseline images and the associated subset of the follow-up images, In response to each of the aforementioned differences in the vascular region exceeding a threshold, The steps include labeling each subset of the baseline images with a significant aneurysm enlargement label, In response to the fact that the respective differences in the vascular region of each subset of the baseline image and the associated subset of the follow-up image fall below a threshold, The steps include labeling each subset of the baseline images with a non-significant aneurysm enlargement label, For each subset of the baseline image, the steps include extracting each set of features from the aortic region, A step of training at least one classifier to classify the set of baseline images based on each set of features by using each of the aforementioned labels as a target, wherein for each baseline image, Using at least one of the classifiers, the steps include classifying each subset of the baseline images based on the set of features as either indicating significant aneurysm enlargement or non-significant aneurysm enlargement, and obtaining predictions for each aneurysm enlargement; A step of updating at least one parameter of the at least one classifier based on each of the aforementioned predictions and each of the aforementioned labels. The training steps include, The steps include: outputting a trained classifier with updated parameters and Methods that include...

39. A system for training at least one classifier to predict the growth of an aneurysm in images acquired by a medical imaging device, A non-temporary computer-readable medium for storing instructions, At least one processor operably connected to the aforementioned non-temporary computer-readable medium and Equipped with, When the at least one processor executes the instruction, For each of the multiple patients Receiving a set of baseline images and a set of follow-up images collected by the medical imaging device in different imaging sessions, wherein each patient has been diagnosed with the aneurysm. The baseline image set and the follow-up image set are compared, and the differences in the vascular region are calculated for each subset of the baseline images and the associated subset of the follow-up images. In response to each of the aforementioned differences in the vascular region exceeding a threshold, Each of the aforementioned subsets of baseline images is labeled with a significant aneurysm enlargement, In response to the fact that the respective differences in the vascular region of each subset of the baseline image and the associated subset of the follow-up image fall below a threshold, Each of the aforementioned subsets of baseline images is labeled with a non-significant aneurysm enlargement label, For each subset of the baseline image, extract each set of features from the aortic region. Training at least one classifier to classify the set of baseline images based on each set of features by using each of the aforementioned labels as a target, wherein for each baseline image, Using the at least one classifier, the respective subsets of the baseline images are classified based on the set of features to indicate either significant aneurysm enlargement or non-significant aneurysm enlargement, and predictions of aneurysm enlargement are obtained. Updating at least one parameter of the at least one classifier based on each of the aforementioned predictions and each of the aforementioned labels. This includes training, Output a trained classifier that includes the updated parameters. A system configured to perform the following actions.

40. A system for predicting the growth of an aneurysm based on at least one image of a given patient who has been previously diagnosed with an aneurysm, A non-temporary computer-readable medium for storing instructions, At least one processor operably connected to the aforementioned non-temporary computer-readable medium and Equipped with, The at least one processor has access to a trained classifier trained to classify patient images as either showing aneurysm enlargement or not showing aneurysm enlargement. When the at least one processor executes the instruction, Receiving a set of images of the body of a given patient, including the aorta, wherein the set of images includes at least one image, and the set of images is collected using a medical imaging device. Using at least one trained segmentation model, segment each of the set of images and extract the vascular region including the lumen of the given patient, Extracting a set of feature quantities from the lumen, including a lumen shape feature quantity that indicates the shape of the lumen, Using the pre-trained classifier, classify each subset of the set of images as either showing aneurysm enlargement or not showing aneurysm enlargement based on at least the set of features, and obtain a set of classified images for the given patient. To predict whether the patient will have an aneurysm enlargement based at least on the classified set of images, Outputting the aforementioned prediction A system configured to perform the following actions.

Images