Method and system for generating 3D parametric meshes of anatomical structures - Patent Application 20070122999

Abstract

A method and system are provided for generating a 3D parametric mesh of a patient's anatomy and storing multi-domain data therein. A plurality of anatomical segments resulting from segmenting a set of patient images are received. A 3D mesh including a plurality of concentric 3D mesh layers is received, each of the concentric 3D mesh layers including the same predetermined number of nodes. Node sets in the 3D mesh corresponding to the respective anatomical segments are determined to obtain respective correspondence rules therebetween. The node sets are encoded with respective feature sets from the respective anatomical segments by using the correspondence rules to obtain the 3D parametric mesh. Each node of the node set in the 3D parametric mesh is associated with a respective plurality of feature channels including the feature sets.

Description

[Technical Field]

[0001] The present technology relates to the field of medical imaging, and more precisely to a method and system for generating a 3D parametric mesh of an anatomical structure, such as the aorta, in order to encode multi-domain features of the structure. [Background technology]

[0002] Post-processing of vascular imaging helps to quantify multiple clinically relevant variables, enhancing the diagnostic and prognostic utility of the scan to the interpreting radiologist (reader) or referring physician. Specifically, this collection of variables generated from the images constitutes a more comprehensive assessment of vascular function, resulting in a more holistic assessment of the patient's health status and more patient-specific treatment approaches and risk assessments.

[0003] The quantified variables can come from multiple physics domains, such as structural mechanics (e.g., geometry, shape, and deformation), fluid mechanics (e.g., luminal flow), or descriptive variables obtained directly from image processing (e.g., image intensity-related variables). Each of these domains must be processed according to a specific data format, such as a shell mesh, a 3D solid mesh, or array-like data. These data formats can be provided with different data densities from multiple domains, different patients, and often from subsequent scans performed on the same patient. These differences in data formats and local data densities hinder the ability to systematically report all available information from the various domains as independent or combined variables for each region of the aorta.

[0004] Furthermore, when training diagnostic or prognostic machine learning models, heterogeneity in data formats forces data scientists to define multiple models with more parameters, this latter aspect increasing the number of subjects required to adequately train generalizable diagnostic and predictive models. Summary of the Invention

[0005] The objective of the present technology is to ameliorate at least some of the inconveniences present in the prior art. One or more embodiments of the present technology may provide and / or expand a range of approaches and / or methods for achieving the goals and objectives of the technology.

[0006] One or more embodiments of the present technology are developed based on the developer's recognition that prognostic and diagnostic applications require a large number of variables or biomarkers to be obtained from multiple physical domains and descriptive representations, which may require data obtained from multiple imaging modalities, a combination of models, simulations, and other computational techniques. Each of these variables can be obtained via a different data structure with different data densities and different data formats for the region of the anatomy under investigation. However, these various representations may not be ideal for clinical reporting and training machine learning models.

[0007] The developers of this technology propose a solution that allows homogenizing data formats and domain data densities across data domains, between and across patients, facilitating multi-domain reporting and modeling for machine learning applications.

[0008] One or more embodiments of the present technology provide an anatomically relevant meshing strategy to produce homogenized data across multiple modalities and scans. Parametric meshes generated using the present disclosure allow data from all data types, ranging from shell and solid meshes to array-like data including pixel-specific data, to be stored in stackable layers that rely on a single type of data encoding and are easy to interpret and utilize for training more compact machine learning-based models.

[0009] One or more embodiments of the present technology provide a 3D parametric mesh comprising multiple concentric three-dimensional (3D) mesh layers, each of which can be interpreted as a separate 3D mesh representing a different interior or exterior layer of an anatomical structure of interest, such that multi-domain information regarding the inner layer (e.g., centerline geometry), outer layer (e.g., wall geometry, strain), and middle layer (e.g., blood flow) of the anatomical structure can be stored and visually represented in the 3D parametric mesh.

[0010] One or more embodiments of the present method and system convert multiple vessel-specific data types into multi-channel, anatomically relevant, stackable images that can be used to train diagnostic and prognostic artificial intelligence-based models, as well as provide organized, intuitive multi-domain reports to medical professionals.

[0011] One or more embodiments of the present technology enable modular modeling for diagnostic and prognostic purposes, leveraging each available domain of information. Because all models rely on the same data types, weights can be optionally shared or very minimally retrained when new information domains are introduced. Modular modeling makes it possible to retrain new architectures or for new tasks by utilizing fewer weights (parameters) and requiring less retraining of these weights. Consequently, this makes it easier to obtain generalizable models starting with a smaller number of vascular scans.

[0012] Accordingly, one or more embodiments of the present technology are directed to methods and systems for generating parametric meshes of anatomical structures.

[0013] According to a broad aspect of the present technology, there is provided a method for generating a 3D parametric mesh of an anatomical structure and storing multi-domain data therein, the method being executed by at least one processor. The method includes: receiving a plurality of anatomical segments of at least some of the anatomical structures within a given patient body obtained from segmentation of an image set acquired by a medical imaging device, the image set including at least one image of at least some of the anatomical structures within the given patient body; receiving a 3D mesh for representing the anatomical structures, the 3D mesh including a plurality of concentric 3D mesh layers, each of the plurality of concentric 3D mesh layers including the same predetermined number of nodes; determining at least one distinct set of nodes in the 3D mesh corresponding to at least one distinct anatomical segment of the plurality of anatomical segments and obtaining a distinct correspondence rule therebetween; and encoding the at least one set of nodes in the 3D mesh with a distinct set of features from the at least one distinct anatomical segment using the correspondence rule to obtain a 3D parametric mesh, wherein each node of the at least one set of nodes in the 3D parametric mesh is associated with a distinct plurality of feature channels including a distinct feature set.

[0014] In one or more embodiments of the method, at least a subset of nodes of the at least one set of nodes are located on different concentric 3D mesh layers.

[0015] In one or more embodiments of the method, encoding at least one node set of the 3D mesh with a distinct feature set from at least one distinct anatomical segment using a correspondence rule to obtain a 3D parametric mesh includes determining a distinct feature set from biomarkers in the at least one distinct anatomical segment using a distinct correspondence rule, and assigning each of the at least one distinct node set with a distinct feature set from the at least one distinct anatomical segment.

[0016] In one or more embodiments of the method, the method further includes receiving a domain representation including individual biomarkers associated with anatomical structures within a given patient body; determining at least one other individual set of nodes in the 3D parametric mesh corresponding to at least one other region in the domain representation to obtain another individual correspondence rule, wherein at least a subset of the other individual second set of nodes is located on a different concentric 3D mesh layer; and encoding the at least one other individual set of nodes in the 3D parametric mesh with another feature set based on the individual biomarkers using the other individual correspondence rule, wherein each node of the at least one other individual set of nodes in the 3D parametric mesh is associated with a separate plurality of feature channels including the other feature set.

[0017] In one or more embodiments of the method, each individual node is further associated with at least one time frame for representing the 3D parametric mesh in time.

[0018] In one or more implementations of the method, each distinct concentric 3D mesh layer is represented as a distinct multidimensional array, and the position of a given node on the distinct 3D mesh layer corresponds to the position of a given node in the distinct multidimensional array.

[0019] In one or more implementations of the method, the multiple feature channels for each node of a separate 3D mesh layer are represented as a separate node array, with each cell of the separate node array corresponding to a separate feature channel of the multiple feature channels.

[0020] In one or more implementations of the method, the domain representation includes another mesh that is different from the 3D parametric mesh, and the other individual correspondence rule includes determining a mapping between nodes in the other mesh and nodes in the 3D parametric mesh.

[0021] In one or more embodiments of the method, the other mesh comprises one of a polygonal mesh, the polygonal mesh comprising one of a triangular mesh, a quadrilateral mesh, a convex polygonal mesh, a concave polygonal mesh, and a polygonal mesh with holes.

[0022] In one or more embodiments of the method, the domain representation includes a structural mechanics representation, and the individual biomarkers include structural mechanics biomarkers, which include at least one of pressure values, strain values, and deformation values.

[0023] In one or more embodiments of the method, the domain representation includes a fluid dynamics representation and the individual biomarkers include at least one of a blood flow value and a shear stress value.

[0024] In one or more embodiments of the method, the domain representation comprises a descriptive variable representation and the individual biomarkers comprise at least one of geometric data values ​​and image data values.

[0025] In one or more embodiments of the method, the anatomical structure includes the aorta of a given patient.

[0026] In one or more embodiments of the method, the plurality of anatomical segments includes the lumen and the aortic wall.

[0027] In accordance with a broad aspect of the present technology, there is provided a system for generating a 3D parametric mesh of an anatomical structure and storing multi-domain data therein, the system comprising at least one processor and a non-transitory storage medium operatively connected to the at least one processor, the non-transitory storage medium storing computer-readable instructions. The at least one processor, when executing the computer-readable instructions, is configured to: receive a plurality of anatomical segments of at least some structures of an anatomical structure within a given patient body obtained from segmentation of an image set acquired by a medical imaging device, the image set including at least one image of at least some structures of an anatomical structure within the given patient body; receive a 3D mesh for representing the anatomical structures, the 3D mesh including a plurality of concentric 3D mesh layers, each of the plurality of concentric 3D mesh layers including the same predetermined number of nodes; determine at least one distinct set of nodes in the 3D mesh corresponding to at least one distinct anatomical segment of the plurality of anatomical segments to obtain a distinct correspondence rule therebetween; and encode the at least one set of nodes of the 3D mesh with a distinct set of features from the at least one distinct anatomical segment using the correspondence rule to obtain a 3D parametric mesh, wherein each node of the at least one set of nodes in the 3D parametric mesh is associated with a distinct plurality of feature channels including a distinct feature set.

[0028] In one or more embodiments of the system, at least a subset of nodes of the at least one set of nodes are located on different concentric 3D mesh layers.

[0029] In one or more embodiments of the system, encoding at least one set of nodes of the 3D mesh with a distinct set of features from at least one distinct anatomical segment using a correspondence rule to obtain a 3D parametric mesh includes determining a distinct set of features from biomarkers in the at least one distinct anatomical segment using a distinct correspondence rule, and assigning each of the at least one distinct set of nodes with a distinct set of features from the at least one distinct anatomical segment.

[0030] In one or more embodiments of the system, the at least one processor is further configured to: receive a domain representation including individual biomarkers associated with anatomical structures within a given patient body; determine at least one other individual set of nodes in the 3D parametric mesh corresponding to at least one other region in the domain representation to obtain another individual correspondence rule, wherein at least a subset of the other individual second set of nodes is located on a different concentric 3D mesh layer; and encode the at least one other individual set of nodes in the 3D parametric mesh with another feature set based on the individual biomarkers using the other individual correspondence rule, wherein each node of the at least one other individual set of nodes in the 3D parametric mesh is associated with a plurality of individual feature channels including the other feature set.

[0031] In one or more embodiments of the system, each individual node is further associated with at least one time frame for representing the 3D parametric mesh in time.

[0032] In one or more embodiments of the system, each individual concentric 3D mesh layer is represented as an individual multidimensional array, and the position of a given node on the individual 3D mesh layer corresponds to the position of a given node in the individual multidimensional array.

[0033] In one or more embodiments of the system, the multiple feature channels for each node of a separate 3D mesh layer are represented as a separate node array, with each cell of the separate node array corresponding to a separate feature channel of the multiple feature channels.

[0034] In one or more embodiments of the system, the domain representation includes another mesh that is different from the 3D parametric mesh, and another separate correspondence rule includes determining a mapping between nodes in the another mesh and nodes in the 3D parametric mesh.

[0035] In one or more embodiments of the system, the other mesh includes one of a polygonal mesh, the polygonal mesh including one of a triangular mesh, a quadrilateral mesh, a convex polygonal mesh, a concave polygonal mesh, and a polygonal mesh with holes.

[0036] In one or more embodiments of the system, the domain representation includes a structural mechanics representation, and the individual biomarkers include structural mechanics biomarkers, which include at least one of pressure values, strain values, and deformation values.

[0037] In one or more embodiments of the system, the domain representation includes a fluid dynamics representation and the individual biomarkers include at least one of a blood flow value and a shear stress value.

[0038] In one or more embodiments of the system, the domain representation includes a descriptive variable representation and the individual biomarkers include at least one of geometric data values ​​and image data values.

[0039] In one or more embodiments of the system, the anatomical structure includes the aorta of a given patient.

[0040] In one or more embodiments of the system, the plurality of anatomical segments includes the lumen and the aortic wall.

[0041] Terms and Definitions In the context of this specification, a "server" is a computer program that runs on appropriate hardware and can receive requests (e.g., from electronic devices) over a network (e.g., a communications network) and carry out those requests or cause those requests to be carried out. The hardware can be a physical computer or a physical computer system, but need not be either, in the context of the present technology. In this context, the use of the term "server" is not intended to imply that all tasks (e.g., received instructions or requests) or any particular task will be received, executed, or caused to be executed by the same server (i.e., the same software and / or hardware). It is intended to mean that any number of software elements or hardware devices may be involved in receiving / sending, executing, or causing to be executed any task or request, or the result of any task or request. All of this software and hardware may be one server or multiple servers, both of which are included in the terms "at least one server" and "server."

[0042] In the context of this specification, a "computing device" is any computing device or computer hardware capable of executing software appropriate for the relevant task at hand. Thus, some (non-limiting) examples of electronic devices include general-purpose personal computers (desktops, laptops, netbooks, etc.), mobile computing devices, smartphones, and tablets, as well as network equipment such as routers, switches, and gateways. Note that a computing device in this context does not exclude acting as a server to other computing devices. The use of the expression "computing device" does not exclude multiple computing devices from being used to receive / send, execute, or cause to be executed a task or request, or the results of a task or request, or any method steps, described herein. In the context of this specification, a "client device" refers to any of a variety of end-user client computing devices associated with a user, such as a personal computer, tablet, smartphone, etc.

[0043] In the context of this specification, unless expressly specified otherwise, computer system refers to, but is not limited to, an "electronic device," "client device," "computing device," "operating system," "system," "computer-based system," "computer system," "network system," "network device," "controller unit," "monitoring device," "control device," "server," and / or any combination thereof appropriate to the task at hand.

[0044] In the context of this specification, the expression "computer-readable storage medium" (also referred to as "storage medium" and "storage") is intended to include non-transitory media of whatever nature and type, including but not limited to RAM, ROM, disks (CD-ROM, DVD, floppy disk, hard drive, etc.), USB keys, solid state drives, tape drives, etc. A computer information storage medium may be formed by combining multiple components, including two or more media components of the same type and / or two or more media components of different types.

[0045] In the context of this specification, a "database" is any structured collection of data, regardless of its particular structure, database management software, or computer hardware on which the data is stored, implemented, or otherwise made available. A database may reside on the same hardware as the processes that store or utilize the information stored in the database, or it may reside on separate hardware, such as a dedicated server or multiple servers.

[0046] In the context of this specification, the expression "information" includes information of any nature or type that can be stored in a database, and thus includes, but is not limited to, audiovisual works (images, films, sound recordings, presentations, etc.), data (location data, numerical data, etc.), text (opinions, comments, questions, messages, etc.), documents, spreadsheets, word lists, etc.

[0047] In the context of this specification, unless expressly specified otherwise, a "representation" of an information element may be the information element itself, or it may be a pointer, reference, link, or other indirection mechanism that allows the recipient of the representation to locate a network, memory, database, or other computer-readable medium where the information element can be obtained. For example, a representation of a document may include the document itself (i.e., its contents), a unique document descriptor that identifies the file with respect to a particular file system, or other means that directs the recipient of the representation to a network location, memory address, database table, or other location where the file can be accessed. As those skilled in the art will recognize, the degree of precision required in such a representation depends on the degree of prior understanding of any interpretation to be given to the information exchanged between the sender and recipient of the representation. For example, if, prior to communication between the sender and recipient, a representation of an information element is known to take the form of a database key for an entry in a particular table of a given database that contains the information element, then merely transmitting the database key can effectively convey the information element to the recipient, even though the information element itself has not been transmitted between the sender and recipient of the representation.

[0048] In the context of this specification, the expression "communications network" is intended to include telecommunications networks such as computer networks, the Internet, telephone networks, Telex networks, TCP / IP data networks (e.g., WAN networks, LAN networks, etc.), etc. The term "communications network" includes wired networks or direct-wired connections, wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media, and combinations of any of the above.

[0049] In the context of this specification, words such as “first,” “second,” and “third” are used as adjectives solely to distinguish between the nouns they modify, and not to describe a particular relationship between those nouns. Thus, for example, it should be understood that the use of the terms “first server” and “third server” is not intended to suggest a particular order, type, chronology, hierarchy, or (e.g.) ranking between servers, nor is their use (in and of itself) intended to suggest that a “second server” must necessarily be present in any given situation. Furthermore, as described elsewhere herein, reference to a “first” element and a “second” element does not exclude the two elements from being the same actual, real-world element. Thus, for example, in some cases, the “first” server and the “second” server may be the same software and / or hardware, while in other cases, they may be different software and / or hardware.

[0050] Each embodiment of the present technology will have at least one, but not necessarily all, of the above-described objects and / or aspects, and it will be understood that some aspects of the present technology that arise from seeking to achieve the above-described object may not meet that object and / or may meet other objects not specifically set forth herein.

[0051] Additional and / or alternative features, aspects, and advantages of embodiments of the present technology will become apparent from the following description, the accompanying drawings, and the appended claims.

[0052] For a better understanding of the present technology, as well as other aspects and further features thereof, reference is made to the following description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0053] [Figure 1]1 shows a schematic diagram of an electronic device in accordance with one or more non-limiting embodiments of the present technology. [Figure 2] 1 shows a schematic diagram of a communication system in accordance with one or more non-limiting embodiments of the present technology. [Figure 3] FIG. 1 shows a schematic diagram of a parametric mesh generation procedure in accordance with one or more non-limiting embodiments of the present technique. [Figure 4] FIG. 1 shows a non-limiting example of a perspective view of a visual rendering of a first parametric mesh taken from the anterior left side of the aorta including the iliac arteries, in accordance with one or more non-limiting embodiments of the present technology. [Figure 5A] 10 shows a perspective view of a visual rendering of the parametric mesh of FIG. 4 with the top portion removed according to line 11 to show multiple concentric 3D mesh layers. [Figure 5B] 5B shows a detailed view of a visual rendering of the parametric mesh of FIG. 5A, along with a schematic view of a selected node and its multiple feature channels. [Figure 6] A top view of a visual rendering of the parametric mesh 400 of FIG. 5 with the upper portion removed according to line 11 is shown, along with a schematic diagram of a selected node and its multiple feature channels. [Figure 7] FIG. 10 shows a perspective view of a visual rendering of a second parametric mesh taken from the anterior left side of the aorta and iliac arteries, in accordance with one or more non-limiting embodiments of the present technology. [Figure 8] FIG. 10 shows a schematic diagram of a first user interface illustrating a visual representation of a third parametric mesh of the aorta including the iliac arteries and a corresponding outer layer arrangement with user interface components for navigating information within the third parametric mesh. [Figure 9A]FIG. 10 shows a schematic diagram of a second user interface showing a visual representation of a fourth parametric mesh of an aorta with iliac arteries with concentric mesh layers between the lumen and the wall selected and a corresponding layer arrangement with user interface components for navigating information within the fourth parametric mesh, the second user interface being shown in accordance with one or more non-limiting embodiments of the present technology. [Figure 9B] 9B shows a schematic diagram of the second user interface of FIG. 9A with the core concentric mesh layer selected in the user interface component. [Figure 10A] 1 shows a flowchart of a method for generating a parametric mesh performed in accordance with one or more non-limiting embodiments of the present technique. [Figure 10B] 1 shows a flowchart of a method for generating a parametric mesh performed in accordance with one or more non-limiting embodiments of the present technique. DETAILED DESCRIPTION OF THE INVENTION

[0054] The examples and conditional language described herein are intended primarily to aid the reader in understanding the principles of the present technology, and are not intended to limit the scope of the present technology to such specifically described examples and conditions. Those skilled in the art will understand that, although not explicitly described or shown herein, they can devise various configurations that embody the principles of the present technology and are within its spirit and scope.

[0055] Furthermore, to aid in understanding, the following description may describe a relatively simplified implementation of the technology, as those skilled in the art will appreciate that various implementations of the technology can be more complex.

[0056] In some cases, what are believed to be useful examples of modifications to the technology may also be described. This is done merely to aid in understanding and, again, does not define the scope or delimit the technology. These modifications are not an exhaustive list, and one of ordinary skill in the art may nonetheless make other modifications while remaining within the scope of the technology. Furthermore, if examples of modifications are not described, it should not be construed that modifications are not possible and / or that what is described is the only way to implement that element of the technology.

[0057] Furthermore, all statements herein reciting principles, aspects, and implementations of the present technology, as well as specific examples thereof, are intended to encompass both structural and functional equivalents, whether now known or developed in the future. Thus, for example, those skilled in the art will understand that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the present technology. Similarly, flowcharts, flow diagrams, state transition diagrams, pseudocode, and the like, may be substantially embodied on computer-readable media and will be understood to represent various processes that may be executed by such a computer or processor, whether or not such a computer or processor is explicitly shown.

[0058] The functionality of the various elements shown in the figures may be provided through the use of dedicated hardware and hardware capable of executing software in conjunction with appropriate software, including any functional blocks labeled as a "processor" or "graphics processing unit." When provided by a processor, the functionality may be provided by a single dedicated processor, by a single shared processor, or by multiple individual processors, some of which may be shared. In some non-limiting implementations of the present technology, the processor may be a general-purpose processor such as a central processing unit (CPU), or a processor dedicated to a specific purpose such as a graphics processing unit (GPU). Furthermore, explicit use of the terms "processor" or "controller" should not be construed to refer only to hardware capable of executing software; it may implicitly include, but is not limited to, digital signal processor (DSP) hardware, network processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), read-only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage. Other hardware, conventional and / or custom, may also be included.

[0059] Software modules, or modules that are simply implied to be software, may be represented herein as any combination of flowchart elements or other elements showing the execution of process steps and / or textual descriptions. Such modules may be executed by explicitly or implicitly shown hardware.

[0060] With these fundamentals in mind, we now consider some non-limiting implementations of the present technology.

[0061] Referring to FIG. 1, a schematic diagram of a computing device 100 suitable for use in some non-limiting implementations of the present technology is shown.

[0062] Computing Devices Computing device 100 comprises various hardware components, including one or more single-core or multi-core processors, collectively referred to as processor 110, a graphics processing unit (GPU) 111, a solid-state drive 120, random access memory 130, a display interface 140, and an input / output interface 150.

[0063] Communication between the various components of computing device 100 may be enabled by one or more internal and / or external buses 160 (e.g., PCI bus, Universal Serial Bus, IEEE 1394 "Firewire" bus, SCSI bus, Serial ATA bus, etc.) to which the various hardware components are electronically coupled.

[0064] Input / output interface 150 may be coupled to touchscreen 190 and / or to one or more internal and / or external buses 160. Touchscreen 190 may be part of a display. In some implementations, touchscreen 190 is a display. Touchscreen 190 may also be referred to as screen 190. In the implementation shown in FIG. 2, touchscreen 190 includes touch hardware 194 (e.g., pressure-sensitive cells embedded in a layer of the display that enable detection of physical interaction between a user and the display) and a touch input / output controller 192 that enables communication with display interface 140 and / or one or more internal and / or external buses 160. In some implementations, input / output interface 150 may be connected to a keyboard (not shown), a mouse (not shown), or a trackpad (not shown), allowing a user to interact with computing device 100 in addition to or instead of touchscreen 190.

[0065] According to an embodiment of the present technology, the solid-state drive 120 stores program instructions suitable for being loaded into the random access memory 130 and executed by the processor 110 and / or GPU 111 to generate a parametric mesh. For example, the program instructions may be part of a library or an application.

[0066] Computing device 100 may be implemented in the form of a server, a desktop computer, a laptop computer, a tablet, a smartphone, a personal digital assistant, or any device that can be configured to implement the present technology, as would be understood by one skilled in the art.

[0067] system Referring to FIG. 2 , a schematic diagram of a communication system 200 is shown, referred to as system 200, suitable for implementing a non-limiting embodiment of the present technology. It should be clearly understood that the illustrated system 200 is merely an exemplary embodiment of the present technology. Accordingly, the following description thereof is intended only as a description of an exemplary example of the present technology. This description is not intended to define the scope or delimit the present technology. In some cases, what are believed to be useful examples of modifications to system 200 are described below. This is done merely to facilitate understanding and, again, is not intended to define the scope or delimit the present technology. These modifications are not an exhaustive list, and other modifications are likely possible, as would be understood by one skilled in the art. Furthermore, if a modification is not made (i.e., if no example of a modification is mentioned), it should not be interpreted as meaning that the modification is not possible and / or that what is described is the only way to implement that element of the present technology. As would be understood by one skilled in the art, this is likely not the case. Furthermore, it will be understood that system 200 may, in certain instances, provide a simple implementation of the present technology, and in such cases, they are presented in this manner to aid in understanding. As will be appreciated by those skilled in the art, various implementations of the present technology may be more complex.

[0068] The system 200 includes, among other things, one or more medical imaging devices 210, a server 230, and a database 235 coupled via a communications network 220 by individual communications links 225 (not separately numbered).

[0069] In one or more embodiments, at least a portion of the system 200 implements Picture Archiving and Communication System (PACS) technology.

[0070] One or more medical imaging devices 210 are operated by a user (eg, a doctor or technician) to acquire medical images of a given patient's body.

[0071] Medical Imaging Equipment The one or more medical imaging devices 210 are referred to herein as medical imaging devices 210 .

[0072] The medical imaging device 210 is configured, among other things, to (i) acquire one or more images of an anatomical structure of interest of a given patient according to acquisition parameters, and (ii) transmit the images to the workstation computer 215 and / or the server 230.

[0073] The medical imaging device 210 may comprise one of an X-ray device, a computed tomography (CT) scanner, a magnetic resonance imaging (MRI) scanner, ultrasound, or the like.

[0074] In some implementations of the present technology, the medical imaging device 210 may include multiple medical imaging devices such as, but not limited to, an X-ray device, a computed tomography (CT) scanner, a magnetic resonance imaging (MRI) scanner, ultrasound (including 2D or 3D ultrasound), positron emission tomography (PET), single photon emission computed tomography (SPECT), and the like.

[0075] The medical imaging device 210 may be configured with specific acquisition parameters for acquiring images of a patient including one or more anatomical structures of interest.

[0076] As a non-limiting example, in one or more embodiments in which the medical imaging device 210 is implemented as a CT scanner, a CT protocol may be used that includes a preoperative retrospective gated multi-detector CT (MDCT—64-row multi-slice CT scanner) that may use variable dose radiation to capture RR intervals.

[0077] As another non-limiting example, in one or more embodiments in which the medical imaging procedure includes an MRI scanner, the MR protocol can include steady-state T2-weighted fast field echo (TE=2.6 ms, TR=5.2 ms, flip angle 110 degrees, fat suppression (SPIR), echo time 50 ms, maximum 25 cardiac phases, matrix 256x256, acquisition voxel MPS (measurement, phase, and slice encoding directions) 1.56 / 1.56 / 3.00 mm, and reconstruction voxel MPS 0.78.

[0078] In one or more alternative embodiments, the medical imaging device 210 may include or be connected to a workstation computer 215 for controlling, among other things, acquisition parameters and transmission of image data.

[0079] Workstation Computer The workstation computer 215 is configured to, among other things, (i) control acquisition parameters of the medical imaging device to perform medical imaging, (ii) receive and process images from the medical imaging device 210, and (iii) transmit images to the server 230.

[0080] The workstation computer 215 is configured to control acquisition parameters of the medical imaging device 210 .

[0081] The workstation computer 215 may receive images in raw format from the medical imaging device 210 and perform tomographic reconstruction using known algorithms and software.

[0082] Implementations of workstation computers 215 are known in the art. Workstation computers 215 may be implemented as computing device 100 or as components of computing device 100, such as processor 110, graphics processing unit (GPU) 111, solid-state drive 120, random access memory 130, display interface 140, and input / output interface 150.

[0083] In one embodiment, the workstation computer 215 is configured according to the Digital Imaging and Communications in Medicine (DICOM) standard for communication and management of medical imaging information and related data.

[0084] The workstation computer 215 is connected to a server 230 via a communications network 220 by a communications link (not numbered).

[0085] In one or more alternative embodiments, the workstation computer 215 may be provided along with the medical imaging device 210. In one or more other embodiments, the workstation computer 215 may be implemented as a mobile device such as a smartphone or tablet.

[0086] In one or more embodiments, the medical imaging device 210, along with other electronic devices such as a server 230, is part of a Picture Archiving and Communication System (PACS) for communicating and managing medical imaging information and related data.

[0087] server The server 230 is configured to, among other things, (i) initialize a mesh of one or more anatomical structures according to a set of mesh parameters; (ii) receive an image set of a given patient including at least a portion of one or more anatomical structures, the image set being acquired by the medical imaging device 210; (iii) segment the image set to obtain a set of segmented images including a plurality of anatomical segments; (iv) receive a plurality of domain representations of one or more anatomical structures including individual biomarkers; (v) determine individual correspondence rules between each domain representation and the initial mesh; and (vi) encode the initial mesh with each individual feature set obtained from the biomarkers in the domain representation based on the individual correspondence rules to obtain a parametric mesh, wherein each node of the parametric mesh includes multiple feature channels including the individual feature sets.

[0088] In some implementations, the server 230 may have access to a set of machine learning (ML) models 250 to perform some of the aforementioned processes.

[0089] How server 230 is configured is described in more detail later in this specification.

[0090] Server 230 may be implemented as a conventional computer server and may include some or all of the components of computing device 100 shown in FIG. 2. In one or more example implementations of the present technology, server 230 may be implemented as a Dell™ PowerEdge™ server running the Microsoft™ Windows Server™ operating system. Of course, server 230 may be implemented with any other suitable hardware and / or software and / or firmware, or combination thereof. In the illustrated non-limiting implementation of the present technology, server 230 is a single server. In alternative non-limiting implementations of the present technology, the functionality of server 230 may be distributed and implemented across multiple servers (not shown).

[0091] Implementations of server 230 are well known to those skilled in the art. However, briefly described, server 230 comprises a communication interface (not shown) structured and configured to communicate with various entities (e.g., workstation computer 215 and other devices that may be connected to network 220) over communication network 220. Server 230 further comprises at least one computer processor (e.g., processor 110 or GPU 111 of computing device 100) operably coupled to the communication interface and structured and configured to execute the various processes described herein.

[0092] In one or more embodiments, server 230 may be implemented as computing device 100 or may include components of computing device 100 such as processor 110, graphics processing unit (GPU) 111, solid-state drive 120, random access memory 130, display interface 140, and input / output interface 150.

[0093] It will be appreciated that server 230 can provide the output of one or more processing steps to another electronic device for display, review, and / or problem resolution. As a non-limiting example, server 230 can transmit images, calculations, results, machine learning parameters for display on a client device configured similarly to computing device 100, such as a smartphone, tablet, or the like.

[0094] The server 230 has access to a set of ML models 250 .

[0095] Machine Learning (ML) Models The set of ML models 250 includes, among other things, a set of segmented ML models 260.

[0096] The ML model will be referred to hereafter as the model.

[0097] Each of the set of models 250 is parameterized by, among other things, model parameters and hyperparameters.

[0098] Model parameters are the configuration variables of the model used to perform predictions and are estimated or learned from training data, i.e., the coefficients are selected during training based on an optimization strategy to output predictions. Hyperparameters are the configuration variables of the model that determine the structure of the initial model and how it is trained.

[0099] It will be appreciated that the number of model parameters to initialize will depend, among other things, on the type of model (e.g., classification or regression), the model architecture (e.g., DNN, SVM, etc.), and the model's hyperparameters (e.g., number of layers, type of layers, number of neurons in the NN).

[0100] In one or more embodiments, the hyperparameters include one or more of the number of hidden layers and units, the optimization algorithm, the learning rate, the momentum, the activation function, the mini-batch size, the number of epochs, and dropout.

[0101] Segmentation Model The set of segmentation models 260 includes one or more segmentation models.

[0102] The segmentation model 260 is configured to perform segmentation of anatomical structures in images acquired by a medical imaging modality, such as the medical imaging device 210 .

[0103] In one or more embodiments, the set of segmentation models 260 is configured to detect (ie, define) all boundaries within an image containing anatomical structures and to identify (ie, classify) various tissue types.

[0104] In one or more embodiments, when the anatomical structure of interest includes an aortic region, the set of segmentation models 260 is configured to segment the outer wall of the aorta, the inner wall of the aorta, the lumen, and the intraluminal thrombus (ILT). Thus, the segmentation model 260 can classify each pixel in the image as one of the outer wall of the aorta, the inner wall of the aorta, the lumen, the intraluminal thrombus (ILT), and the background.

[0105] In one or more embodiments, the set of segmentation models 260 refers to multiple segmentation models 260, each configured to perform a particular segmentation task. As a non-limiting example, the segmentation models 260 may include a first segmentation model configured to perform foreground and background segmentation, a second segmentation model configured to perform semantic segmentation of the lumen within the aorta, and a third model configured to perform pathological tissue classification (e.g., classification of calcified and non-calcified tissue within the aortic wall and intraluminal thrombus, if present). Non-limiting examples of such segmentation models are described in International Patent Application No. PCT / IB2022 / 051558, entitled "METHOD AND SYSTEM FOR SEGMENTING AND CHARACTERIZING AORTIC TISSUES," filed February 22, 2022 by the same applicant, the contents of which are incorporated herein by reference.

[0106] In one or more implementations, the set of segmentation models 260 may be composed of convolutional neural network layers (e.g., U-Net or V-Net based), attention-based mechanisms (i.e., transformer-based models such as Vision Transformer (ViT) models), and combinations thereof. It will be appreciated that the set of segmentation models 260 may use an encoder-decoder architecture.

[0107] In one or more embodiments, the set of segmentation models 260 may be based on fully convolutional neural networks (FCNs), generative adversarial networks (GANs), cascade networks, etc.

[0108] In one or more embodiments, the segmentation model 260 has a ResNet-based FCN architecture. Non-limiting examples of ResNets include ResNet50 (50 layers), ResNet101 (101 layers), ResNet152 (152 layers), ResNet50V2 (50 layers with batch normalization), ResNet101V2 (101 layers with batch normalization), and ResNet152V2 (152 layers with batch normalization).

[0109] In one or more alternative embodiments, the set of segmentation models 260 may be implemented based on one of U-Net, V-Net, SegNet, AlexNet, GoogleNet, VGG, DeepLab, Mask R-CNN, etc.

[0110] Database The database 235 is configured to, among other things, (i) store acquisition parameters and data related to the medical imaging device 210, (ii) store images acquired by a medical imaging modality such as the medical imaging device 210, (iii) store data related to a set of ML models 250, including model parameters, hyperparameters, datasets, and outputs, (iv) store data related to meshes, (v) store multiple domain representations including biomarkers, and (vi) generate parametric meshes including all their feature channels.

[0111] Database 235 is configured to store images and videos. In one or more embodiments, the database can store Digital Imaging and Communications in Medicine (DICOM) files, including, for example, DCM and DCM30 (DICOM 3.0) file extensions. Additionally or alternatively, database 235 can store medical image files in Tag Image File Format (TIFF), Digital Storage and Retrieval (DSR) TIFF-based format, and Data Exchange File Format (DEFF) TIFF-based format.

[0112] In one or more embodiments, database 235 may also store ML file formats such as .tfrecords, .csv, .npy, and .petastorm, as well as file formats used to store models such as .pb, .pkl, .pt, or .pth. Database 235 may also store known file formats such as image file formats (e.g., .png, .jpeg, .exif, .bmp, .tiff), video file formats (e.g., .mp4, .mkv, etc.), archive file formats (e.g., .zip, .gz, .tar, .bzip2), document file formats (e.g., .docx, .pdf, .txt), or web file formats (e.g., .html).

[0113] It will be appreciated that the database 235 may store other types of data, such as a validation data set (not shown), a test data set (not shown), and the like.

[0114] communication network In some implementations of the present technology, the communications network 220 is the Internet. In alternative non-limiting implementations, the communications network 220 may be implemented as any suitable local area network (LAN), wide area network (WAN), private communications network, etc. It should be expressly understood that the implementation of the communications network 220 is for illustrative purposes only. How the communications links 225 (not separately numbered) between the workstation computer 215 and / or server 230 and / or another electronic device (not shown) and the communications network 220 is implemented will depend, among other things, on how each of the medical imaging device 210, the workstation computer 215, and the server 230 is implemented.

[0115] Communications network 220 may be used to transmit data packets between workstation computer 215, server 230, and database 235. For example, communications network 220 may be used to transmit requests between workstation computer 215 and server 230.

[0116] Parametric mesh generation procedure Referring to FIG. 3, a schematic diagram of a parametric mesh generation procedure 300 is shown in accordance with one or more non-limiting embodiments of the present technique.

[0117] The goal of the parametric mesh generation procedure 300 is to generate a three-dimensional (3D) parametric mesh that is a structural representation of one or more anatomical structures of a given patient, where each element or node of the parametric mesh encodes, among other things, structural, temporal, functional, and other descriptive information (also called biomarkers) at the corresponding location, which may be acquired using different imaging modalities and / or physics domain representations. How the parametric mesh generation procedure 300 is configured to achieve that goal is described in more detail below.

[0118] The parametric mesh generation procedure 300 includes, among other steps, a mesh initialization procedure 320, an image acquisition procedure 330, a segmentation procedure 340, a multi-domain data acquisition procedure 350, a registration procedure 380, and a parametric mesh encoding procedure 500.

[0119] In one or more embodiments, the parametric mesh generation procedure 300 may be performed by the server 230. In one or more alternative embodiments, the parametric mesh generation procedure 300 may be performed by one or more computing devices in a distributed manner. As a non-limiting example, a first computing device, such as the server 230, may perform at least a portion of the parametric mesh generation procedure 300 (i.e., one of the steps), and one or more other computing devices may perform other portions of the parametric mesh generation procedure 300 (i.e., other of the steps).

[0120] The parametric mesh generation procedure 300 includes a mesh initialization procedure 320 .

[0121] Mesh initialization procedure The mesh initialization procedure 320 is configured to, among other things, (i) receive a mesh parameter set and (ii) generate an initial 3D mesh based on the mesh parameter set. The mesh initialization procedure 320 allows for the specification of mesh dimensions, geometry, and other attributes (e.g., initial conditions or parameters) before the mesh is loaded with patient-specific biomarker data. The mesh initialization procedure 320 serves as a preprocessing step to facilitate subsequent encoding of the mesh with biomarker data for analysis or visualization.

[0122] In the context of the present technology, the mesh initialization procedure 320 is used to initialize a 3D mesh, which is a multidimensional array that provides, among other things, a spatial representation of one or more anatomical structures in the form of a 3D geometric shape including a discrete number of volume elements, also called "elements" or "cells." It will be appreciated that the mesh may provide a discrete spatial and temporal representation of one or more anatomical structures for a given patient. The nodes of the initial 3D mesh are then populated or encoded with single-modal or multi-modal data, i.e., one or more biomarkers, for the same patient, resulting in a 3D parametric mesh, as described below. The 3D parametric mesh may be used to store biomarker data from multiple domains in the form of features and to visually render the features over time.

[0123] The mesh initialization procedure 320 may be performed any time before the registration procedure 380 .

[0124] The mesh initialization procedure 320 receives a set of mesh parameters. The set of mesh parameters may be entered by an operator of the technology via an input / output device, such as a keyboard, touch screen, etc. In one or more embodiments, the set of mesh parameters may be received from another client device.

[0125] The set of mesh parameters may specify the configuration of the mesh for each anatomical structure to be represented and may include one or more of its geometry, number of nodes, type of volume elements, and number of volume elements.

[0126] In one or more embodiments, the one or more anatomical structures represented by the mesh include the aorta. Additionally, the one or more anatomical structures represented by the mesh may include the iliac arteries.

[0127] The mesh parameter set includes a predetermined total number of nodes that form the initial 3D mesh. It will be understood that the number of nodes is not limited and may include, as a non-limiting example, 10,000 nodes. In one or more alternative implementations, a number of predetermined feature channels may be associated with each node.

[0128] The total number of nodes may be set by an operator of the present technology. In some alternative implementations of the present technology, the number of nodes may depend on the number of feature channels, the computational resources (e.g., the storage capacity of the electronic device used to implement the present technology), and the intended use of the 3D parametric mesh.

[0129] In one or more embodiments, the total number of nodes in the 3D mesh as well as the number of nodes in at least one of each concentric 3D mesh layer, each layer parallel to the axial plane, and each layer parallel to the sagittal plane can be predetermined.

[0130] The initial 3D mesh includes or defines multiple concentric 3D mesh layers positioned relative to a centerline. Each concentric 3D mesh layer may be understood to be a separate 3D mesh with its nodes positioned at a separate distance from a separate section of a shared centerline (e.g., a 3D point or line defining the center of the represented anatomical structure). Briefly referring to Figures 5A and 5B, Figure 5B shows the first parametric mesh 400 with the top portion removed to reveal multiple concentric 3D mesh layers 440, which are described in more detail below.

[0131] Returning to Figure 3, multiple concentric 3D mesh layers can be used to represent different boundaries of the anatomical structures, substructures, and / or spaces between them for which information is to be encoded. By way of non-limiting example, the initial 3D mesh can include 3D mesh layers referenced to an outer mesh representing the outer surface of the anatomical structure and / or referenced to an innermost mesh at the centerline of the anatomical structure. It will be appreciated that in one or more alternative embodiments, the 3D mesh can include one or more additional layers outside the anatomical structure.

[0132] The centerline of the 3D mesh may correspond to the centerline of the anatomical structure of interest, as determined during the registration procedure 380. It should be understood that sections of the centerline may extend vertically and horizontally in 3D, and that the centerline generally follows the shape of the anatomical structure being represented.

[0133] For a given fixed ordinate (i.e., a fixed value on the vertical or z-axis) corresponding to an axial (transverse) slice (i.e., parallel to a transverse or axial plane), each concentric 3D mesh layer is sometimes referred to as a two-dimensional (2D) concentric mesh axial layer or 2D mesh axial slice. Referring briefly to Figure 6, an axial view of the parametric mesh 400 is shown showing multiple 2D concentric mesh layers 440, which will be described in more detail herein below.

[0134] Returning to FIG. 3 , it will be appreciated that when the initial 3D mesh is initialized, its 3D visual representation has not yet been generated. This is because the initial 3D mesh has not yet been encoded and populated with data from actual measurements of the patient's anatomical structure(s). Thus, the initial 3D mesh is "generic" or the same for each patient before being encoded with patient-specific data and, optionally, visually represented on a graphical user interface. In one or more alternative embodiments, the initial 3D mesh may be represented in 3D using a generic or "default" shape of the anatomical structure.

[0135] Each concentric 3D mesh layer may be "unwrapped" and represented as a 2D array, with each element in the 2D array corresponding to a distinct node in the respective concentric 3D mesh layer. Each node is associated with or represented by a distinct node array, sometimes referred to as a "feature channel," that encodes node features. Each location in the node array corresponds to a distinct feature that will be encoded in a distinct node of the 3D mesh upon completion of the parametric mesh generation procedure 300.

[0136] Each node of the initial 3D mesh is associated with a distinct node position coordinate. It will be appreciated that the node position coordinates may be expressed in different and equivalent ways depending on the coordinate system used. The node position coordinates are used to reference the nodes of the mesh or associated array and to store and retrieve information.

[0137] In one or more embodiments, a given node may be identified based on the concentric 3D mesh layer in which it is located using a mesh layer coordinate R. The mesh layer coordinate R may be used to identify the particular concentric layer array in which the node information is stored, as each concentric 3D mesh layer array corresponds to a different array of the same size (i.e., the same number of nodes).

[0138] In one or more embodiments, each node may be identified for each concentric 3D mesh layer, and the coordinate M may refer to the node's circumferential position on the concentric 3D mesh layer (i.e., corresponding to its position along the circumference on an axial slice of the 2D mesh), and the coordinate N may refer to the node's longitudinal position (i.e., corresponding to the 2D axial slice to which the node belongs). Each concentric 3D mesh layer may be "unwrapped" and correspond to a 2D array, so that the node coordinates in the 2D array are the same as the node coordinates in the 3D mesh.

[0139] In some embodiments, each node of the 3D mesh is associated with an individual node identifier, which may include one or more numbers to uniquely identify the node for purposes of retrieval and / or storage of information (e.g., in multiple feature channels). It will be understood that the node coordinates are the same for each patient.

[0140] In one or more embodiments, each node is associated with a distinct node position coordinate and a distinct node time coordinate. The node time coordinate may include two or more time coordinates that allow the node to be represented at different discrete points in time. It will be appreciated that in some embodiments of the present technology, a time coordinate (i.e., a time frame) may not be required. Thus, using time coordinates, the 3D mesh can visually represent changes over time (e.g., changes in the geometry of blood vessels during a cardiac cycle).

[0141] In one or more embodiments, each concentric 3D mesh layer can include MxNxP nodes, where M is the number of circumferential points, N is the number of longitudinal points, and P is the number of time frames. It will be understood that a planar section (i.e., a layer) of the mesh corresponds to all nodes of fixed ordinate coordinate N and fixed temporal coordinate O.

[0142] It will be appreciated that a 3D mesh can be represented as a collection of 2D arrays, with each concentric 3D mesh layer corresponding to a respective 2D array of the same size (i.e., the same number of nodes), with each 2D array element corresponding to a node and including a respective array corresponding to the node's feature channel. In one or more embodiments, a time frame may also be encoded in the collection of 2D arrays, in this case, a given collection of 2D arrays (corresponding to the complete 3D mesh). In other embodiments, a time value may be encoded at each node (i.e., each node may correspond to an array with a feature channel for each instant).

[0143] It is important to understand that each node of a patient's 3D mesh encodes the patient's actual anatomical geometry information into the node's feature channels, and the patient-specific 3D mesh is visually rendered using the feature channels that encode the information.

[0144] The mesh initialization procedure 320 outputs an initial 3D mesh.

[0145] In one or more embodiments, the initial 3D mesh is represented as a multidimensional array including a separate concentric layer array representing each concentric 3D mesh layer, the size of each separate concentric layer array corresponding to the number of nodes in the separate concentric 3D mesh layer, and each node including a separate node array for storing node characteristics, each separate node array capable of storing node characteristics at discrete moments in time.

[0146] Image acquisition procedure The image acquisition procedure 330 is configured, among other things, to receive a set of images of the patient's body acquired by the medical imaging device 210 .

[0147] The set of images of the patient's body includes at least one image of the patient's body that is a discrete representation of a signal containing at least a portion of one or more anatomical structures of interest, generated using the medical imaging device 210.

[0148] It will be appreciated that in the case of a 2D domain representation, the image cells are called "pixels" and in the case of a 3D domain representation, the image cells are called "voxels."

[0149] In one or more embodiments, the set of images may be in the form of an image stack.

[0150] It will be appreciated that an image stack comprises a set of consecutive images, also called scrollable slices, as would be expected for cross-sectional studies (e.g., CT / MRI) or time-resolved modalities. As a non-limiting example, an image stack may be provided in DICOM file format.

[0151] In one or more embodiments, the image stack may be in the form of a multi-phase stack, and each phase of the multi-phase stack may correspond to a time instance. As a non-limiting example, each phase in the stack may correspond to a moment in a given patient's cardiac cycle.

[0152] In one or more embodiments, the set of images of the patient's body includes the aorta(s) and / or the iliac arteries.

[0153] In one or more embodiments, one or more anatomical structures in the image set may include the thoracic region (e.g., ascending aorta, aortic arch, descending thoracic aorta) and / or abdominal aortic region (e.g., suprarenal abdominal aorta, infrarenal aorta, renal arteries, lumbar arteries) and iliac arteries (e.g., common iliac arteries, external iliac arteries, internal iliac arteries).

[0154] It will be appreciated that multiple image sets of a given patient's body, each image set corresponding to a different imaging session, may be received by the image acquisition procedure 330. It will be appreciated that a given image set may be selected as a "baseline" set, and image sets acquired during other imaging sessions may be sent to the multi-domain data acquisition procedure 350 and subsequently encoded.

[0155] In one or more other embodiments, the image acquisition procedure 330 can receive multiple image sets, one or more of the image sets being acquired using different medical imaging modalities.

[0156] The image acquisition procedure 330 outputs a set of images.

[0157] Segmentation Procedure The segmentation procedure 340 is configured to, among other things, (i) receive an image set; and (ii) segment the image set to obtain a set of segmented images including a plurality of anatomical segments.

[0158] The segmentation procedure 340 uses manual and / or automatic segmentation methods to obtain a plurality of anatomical segments, which may also be referred to as segmented tissue.

[0159] In one or more other embodiments, the segmentation procedure 340 may use manual segmentation techniques to obtain the segmented image.

[0160] In one or more embodiments, the segmentation procedure 340 obtains one or more anatomical segments using a set of trained segmentation ML models 260 that are trained to segment anatomical structures in images acquired by an imaging device (e.g., medical imaging device 210). For example, the segmentation procedure 340 may output one of a number of classes for a region in the image, including at least one anatomical segment and background.

[0161] As a non-limiting example, the set of segmentation models 260 may be trained to segment aortic tissue in an image.

[0162] In one or more embodiments, the segmentation procedure 340 obtains a segmented aortic region that includes one or more of the aorta and iliac arteries for each image in the image set. In one or more embodiments, the segmented aortic region includes an ROI that includes the lumen, the aortic wall, the ILT (if present), and calcification (if present).

[0163] Additionally, the segmentation procedure 340 can segment the different branches of the abdominal aorta, including the celiac trunk, the superior and inferior mesenteric arteries, the hepatic artery, the splenic artery, the renal arteries, and the iliac arteries.

[0164] In one or more alternative embodiments, the segmentation procedure 340 outputs a set of segmented images, with each pixel labeled with a distinct segmented tissue label.

[0165] In one or more embodiments, the segmentation procedure 340 extracts segmented tissue from the set of images to obtain at least one image per anatomical segment. It will be appreciated that the segmented tissue can be extracted by performing masking.

[0166] In one or more embodiments, the segmentation procedure 340 determines a center point for each of the anatomical segments, which can be used by the registration procedure 380 to determine the 3D centerline of each anatomical segment.

[0167] The segmentation procedure 340 outputs a representation of multiple anatomical segments for each of the set of images.

[0168] Multi-domain data acquisition procedure The multi-domain data acquisition procedure 350 is configured to, among other things, (i) transmit an image set and / or a segmented image set including multiple anatomical segments, and (ii) receive one or more domain representations including biomarkers associated with anatomical structures within the body.

[0169] In one or more embodiments, the multi-domain data acquisition procedure 350 transmits a set of images and / or a set of segmented images so that different domain representations can be generated based on the transmitted set of images and / or multiple segments, each domain representation including biomarkers associated with an anatomical segment.

[0170] In the context of the present technology, a domain representation should be understood to be a discretized 2D and / or 3D representation of one or more anatomical structures of interest of a given patient, which may or may not include a discretized temporal representation. The domain representation may be computed and / or obtained using one or more imaging modalities.

[0171] The domain representation may include or be associated with biomarker values ​​that may provide structural, functional, and temporal information for elements contained in one or more anatomical structures in the domain representation. Non-limiting examples of biomarkers include pixel intensity, pixel location, structural mechanics values ​​(e.g., pressure, strain, stress, etc.), flow-related values ​​(e.g., velocity), and any other descriptive variables.

[0172] Each domain representation may have a different data format and / or data density and / or resolution.

[0173] In one or more embodiments, the given domain representation may be a mesh, such as a polygonal mesh. A polygonal mesh includes vertices, edges, and faces. The faces may include one of triangles (triangle mesh), quadrilaterals (quads), convex polygons (n-gons), concave polygons, and polygons with holes. The mesh may be a 2D or 3D mesh, with or without time discretization.

[0174] In one or more other embodiments, the given domain representation may be a 2D or 3D image.

[0175] In one or more embodiments, the multi-domain data acquisition procedure 350 sends the set of images and / or the set of segmented images to the fluid dynamics procedure 360 ​​.

[0176] Fluid Dynamics Procedure The fluid dynamics procedure 360 ​​is configured to perform a fluid dynamics analysis to generate a fluid dynamics representation that includes fluid dynamics biomarkers.

[0177] It will be appreciated that the fluid dynamics procedure 360 ​​may be performed by one or more other electronic devices and / or the server 230 .

[0178] In one or more embodiments, the fluid dynamics procedure 360 ​​simulates complex flows through a numerical discretization and solution approach using computational fluid dynamics (CFD) to obtain numerical solutions at discrete time / space points in the flow field. In one or more embodiments, the fluid dynamics biomarkers may include variables related to heat transfer or fluid flow.

[0179] In one or more embodiments, the fluid dynamics procedure 360 ​​performs spatial discretization or meshing based on the set of segmented images to divide the geometry into several discrete volume elements or cells, and then performs time discretization. The fluid dynamics procedure 360 ​​establishes boundary conditions, i.e., a set of applied physiological parameters (which may vary over time) that define the physical conditions at the inlets, outlets, and walls. It will be understood that the boundary conditions may be based on patient-specific data, population data, physical models, and / or assumptions. Further properties for the simulation are defined, including blood density and viscosity (i.e., fluid model), initial conditions of the system (e.g., whether the fluid is initially static or moving), time discretization information (time step size and numerical approximation scheme), and / or desired output data (e.g., number of cardiac cycles to simulate). The fluid dynamics procedure 360 ​​uses a CFD solver to solve the Navier-Stokes and continuity equations, incrementally progressing toward convergence to obtain a final solution. Next, the fluid dynamics procedure 360 ​​obtains fluid dynamics biomarkers including the pressure and velocity fields across all elements at each time step. It will be appreciated that additional biomarkers may be calculated and obtained based on the foregoing.

[0180] In one or more embodiments, the fluid dynamics procedure 360 ​​may use volumetric meshes ranging from 2 million to 3.5 million elements in different possible geometries.

[0181] As a non-limiting example, the fluid mechanics procedure 360 ​​performs a sensitivity analysis to obtain a volumetric mesh of approximately 2 million tetrahedral elements, and runs a CFD simulation in ANSYS™ software FLUENT using a semi-implicit solution of the pressure-implicit method (SIMPLE) and a second-order implicit transient formulation of the pressure-coupled equations, assuming laminar blood flow, a time-varying velocity profile based on the flow rate of the descending aorta at the inlet of the fluid domain, and an outflow boundary condition of 50% flow split to the iliac arteries. The rheology model can assume that blood is an isotropic, incompressible, Newtonian fluid assigned a constant density (1060 kg / m3) and dynamic viscosity (0.00319 Pa s). The arterial wall can be assumed to be rigid, and a no-slip condition can be applied at the fluid interface. The fluid mechanics procedure 360 ​​can output fluid dynamics biomarkers for elements at the boundary of the computational domain. In the aforementioned non-limiting example, the fluid dynamics procedure 360 ​​may output biomarkers such as blood velocity and wall shear stress at the boundary of a mesh containing two tetrahedral million elements.

[0182] As a non-limiting example, for the aorta, models such as Windkessel, a distributed model of arterial behavior, a reservoir pressure model, or finite element analysis can be used to obtain fluid dynamic biomarkers.

[0183] The fluid dynamics procedure 360 ​​outputs a fluid dynamics representation that includes the fluid dynamics biomarkers.

[0184] The fluid dynamic representation specifies information related to the spatial and temporal discretization, and the fluid dynamic biomarker specifies several variables and their values ​​for each of the elements in the spatial and temporal discretization.

[0185] The multi-domain data acquisition procedure 350 receives a fluid dynamics representation that includes fluid dynamics biomarkers.

[0186] In one or more embodiments, the multi-domain data acquisition procedure 350 sends the set of images and / or the set of segmented images to the structural mechanics procedure 365 .

[0187] Structural Mechanics Procedures The structural mechanics procedure 365 is configured to perform a structural mechanics analysis to generate a structural mechanics representation that includes structural mechanics biomarkers, which are variables that indicate the structural and mechanical properties of one or more anatomical structures of interest.

[0188] It will be appreciated that the structural mechanics procedure 365 may be performed by one or more other electronic devices and / or the server 230 .

[0189] The structural mechanics procedure 365 uses a set of segmented images containing multiple anatomical segments to generate a structural mechanics representation, which is used to obtain structural mechanics biomarkers.

[0190] The structural mechanics procedure 365 can obtain structural mechanics biomarkers using domain representations that are different from other domain representations (eg, fluid mechanics and descriptive variable representations).

[0191] In one or more embodiments, the structural mechanics procedure 365 performs spatial discretization or meshing based on the set of segmented images to divide the geometry into a number of discrete surface elements or cells, and then performs temporal discretization. The structural mechanics procedure 365 may generate a specific mesh having tetrahedral, triangular, hexahedral, triangular, and / or rectangular elements to calculate unimodal or multimodal structural mechanics biomarkers. The structural mechanics procedure 365 then performs mechanical property characterization based on the generated mesh. It will be appreciated that various methods, including finite element analysis simulation, can be used.

[0192] The structural mechanics representation is used to obtain structural mechanics biomarkers, which may include stress-strain relationships and strength of one or more anatomical structures of interest.

[0193] Structural mechanics biomarkers may include thickness, strain, and stress, as well as other biomarkers derived based on the foregoing.

[0194] In one or more embodiments, the structural mechanics procedure 365 acquires structural mechanics biomarkers including strain, which may further include maximum principal strain, minimum principal strain, circumferential strain, and longitudinal strain, corresponding strain rates, and / or timing of maximum strain.

[0195] As a non-limiting example, the structural mechanics procedure 365 may generate a surface wall mesh of approximately 4000 triangular shell elements and track nodal velocities in a three-dimensional image stack representing the aorta throughout the cardiac cycle. The structural mechanics procedure 365 may then measure nodal displacements based on the nodal velocities and calculate in vivo strains. In this non-limiting example, the structural mechanics procedure 365 uses a domain representation that includes a surface wall mesh of approximately 4000 triangular shell elements, which differs from the initial mesh used by the fluid mechanics procedure 360 ​​and the volumetric mesh of approximately 2 million tetrahedral elements.

[0196] Thus, in this non-limiting example, the structural mechanics procedure 365 uses a domain representation of the surface wall mesh of 4000 triangular elements and calculates strain biomarkers for the elements of the surface wall mesh.

[0197] As another non-limiting example, the structural mechanics procedure 365 can use the segmented aortic lumen and segmented wall received from the segmentation procedure 340 to generate an aortic lumen surface mesh and an aortic wall surface mesh, and calculate the ILT thickness by measuring the average distance between each mesh point of the outer wall mesh and an adjacent point on the lumen surface within a specified radius.

[0198] Non-limiting examples of methods and systems for generating structural mechanics representations and biomarkers, including regional rupture potential (RRP) of blood vessels (also known as regional aortic weakening (RAW)), are described in detail in International Patent Application Publication WO2021 / 059243A1, entitled "METHOD AND SYSTEM FOR DETERMINING REGIONAL RUPTURE POTENTIAL OF BLOOD VESSEL," filed on September 25, 2020, by the same applicant, the contents of which are incorporated herein by reference.

[0199] The structural mechanics representation specifies information related to the spatial and temporal discretization of an anatomical structure(s), and the structural mechanics biomarkers specify several variables and their values ​​for each of the elements in the spatial and temporal discretization.

[0200] In one or more embodiments, the structural mechanics procedure 365 outputs one or more structural mechanics representations, each associated with a distinct structural biomarker. Each of the one or more structural mechanics representations may have a different mesh geometry and distinct structural biomarkers, and therefore may have a different data density.

[0201] The multi-domain data acquisition procedure 350 receives a structural dynamics representation that includes structural dynamics biomarkers.

[0202] Descriptive variable representation procedure The multi-domain data acquisition procedure 350 sends a set of images and / or a segmented image containing multiple segments to a descriptive variable representation procedure 370 to obtain a representation that includes other biomarkers not described above.

[0203] In one or more alternative embodiments, the multi-domain data acquisition procedure 350 may not send images or segments to the descriptive variable representation procedure 370, but may instead acquire one or more descriptive variable representations that include biomarkers that are not based on images or segments. This may be the case, for example, when other domain representations of the same anatomical structure of a given patient are acquired using other types of imaging modalities.

[0204] It will be appreciated that the descriptive variable representation procedure 370 may be performed by one or more other electronic devices and / or the server 230 .

[0205] In one or more embodiments, the descriptive variable representation procedure 370 may receive images of the same patient acquired by one or more other medical imaging devices different from the medical imaging device 210 .

[0206] In one or more embodiments, the one or more other medical imaging devices may include micro-CT, ultrasound, confocal microscopy, focused ion beam scanning electron microscope (FIBSEM), or the like.

[0207] Thus, it will be appreciated that the descriptive variable representation procedure 370 acquires images of at least a portion of the same patient anatomy.

[0208] As a non-limiting example, the descriptive variable representation procedure 370 can receive ultrasound images having Doppler velocities that may have a different resolution and may have a different acquisition angle and view than the set of images acquired by the image acquisition procedure 310.

[0209] The multi-domain data acquisition procedure 350 receives one or more descriptive variable representations that include individual descriptive biomarkers.

[0210] In one or more embodiments, the parametric mesh generation procedure 300 includes an alignment procedure 380 .

[0211] Alignment Procedure The registration procedure 380 is configured to, among other things, (i) receive multiple domain representations from the multi-domain data acquisition procedure 350, (ii) align the multiple domain representations to a common frame of reference, and (iii) calculate individual correspondence rules between the initial mesh and each of the multiple domain representations.

[0212] In the context of the present technology, the registration procedure 380 is used to bring the various relevant representations and modalities into a common frame of reference (i.e., spatial alignment) so that the information they contain can be optimally integrated into a parametric mesh during the parametric mesh encoding procedure 390.

[0213] In one or more embodiments, the registration procedure 380 is configured to receive results of multi-temporal image analysis where images of the same patient were acquired at different times and / or under different health conditions.

[0214] In one or more embodiments, the registration procedure 380 is configured to perform multi-modality image fusion to align images from different modalities acquired by the multi-domain data acquisition procedure 350 .

[0215] In one or more embodiments, the registration procedure 380 is configured to perform dynamic image sequence analysis to stack still images acquired at different time steps from a dynamic image sequence typically used to capture and quantify the motion of anatomical structures such as the respiratory / cardiac systems.

[0216] In one or more embodiments, the registration procedure 380 is configured to perform data interpolation techniques to obtain biomarker values ​​at inter-cell center or inter-cell locations, which may include deterministic and / or statistical interpolation techniques.

[0217] Non-limiting examples of data interpolation techniques include nearest neighbor interpolation, linear interpolation, spline interpolation, polynomial interpolation, Lagrangian interpolation, Gaussian interpolation, Fourier transform, and wavelet transform.

[0218] In one or more embodiments, the registration procedure 380 receives additional data used during the multi-domain data acquisition procedure 350. As a non-limiting example, if a mesh has been generated and modified in pre-processing and post-processing, the modification information may be sent to the registration procedure 380 to make registration easier.

[0219] In one or more embodiments, the registration procedure 380 is configured to determine, for each domain representation of the plurality of domain representations, a respective center point or a respective center line of one or more anatomical structures of interest in the domain representation. It will be appreciated that the center point / center line may be determined using different methods, including manual methods (e.g., by receiving user input), automatic methods, or a combination thereof.

[0220] The registration procedure 380 is configured to determine a separate correspondence rule between the initial mesh and each of the multiple domain representations. In one or more embodiments, the correspondence rule may be determined based on the center points and / or center lines of each domain representation.

[0221] Correspondence rules are determined for regions in the domain representation that correspond to regions (i.e., sets of nodes) in the initial mesh. As a non-limiting example, the domain representation may be an axial ultrasound image, and the registration procedure 380 may determine axial view and node coordinates in the mesh that correspond to regions in the axial ultrasound image. The registration procedure 380 then determines correspondence rules between pixels in the axial ultrasound image and nodes in the mesh.

[0222] The individual correspondence rules may include a function mapping element from a given domain representation to a corresponding element (set of nodes) in the initial 3D mesh, allowing biomarker data associated with elements of the given domain representation to be mapped as features on the initial mesh. In other words, the individual correspondence rules are functions that describe the mapping from an individual coordinate system of the domain representation to the coordinate system of the initial mesh.

[0223] Since the initial 3D mesh and each domain representation may be based on different types of meshes and therefore may have different biomarker data densities for a given anatomical substructure, it turns out that it is necessary to determine the correspondence between structural elements, i.e., the location and number of elements in the individual domain representations that correspond to a given node at a given location in the initial 3D mesh.

[0224] The registration procedure 380 calculates correspondence rules between elements in the initial mesh and elements in a set of segmented images that include multiple anatomical segments. In one or more embodiments, the correspondence rules may be initially determined so that the set of segmented images is used as a "baseline" visual representation of one or more anatomical structures of the patient that are encoded into the 3D parametric mesh.

[0225] The alignment procedure 380 calculates correspondence rules between elements in the initial mesh and elements in the structural mechanics representation. As a non-limiting example, the alignment procedure 380 may determine regions in the initial mesh of 10,000 nodes that correspond to regions in a surface mesh of 4,000 triangular shell elements of the structural mechanics representation and determine correspondence rules between data associated with the triangular elements and the nodes of the mesh. As a non-limiting example, the regions may include nodes located on different concentric 3D mesh layers.

[0226] The alignment procedure 380 calculates correspondence rules between elements in the initial mesh and elements in the fluid dynamics representation. As a non-limiting example, the alignment procedure 380 can determine which regions in the initial mesh of 10,000 nodes in the fluid dynamics representation correspond to which regions in a volumetric mesh of 4,000,000 tetrahedral elements, and determine correspondence rules between data associated with the tetrahedral elements and the nodes of the mesh. As a non-limiting example, the regions may include nodes located on different concentric 3D mesh layers.

[0227] The registration procedure 380 calculates correspondence rules between elements in the initial mesh and elements in the descriptive variable representation. As a non-limiting example, the registration procedure 380 can determine which regions in the initial mesh of 10,000 nodes correspond to which regions of pixels in the ultrasound image and determine correspondence rules between data associated with the pixels and nodes of the mesh. As a non-limiting example, the regions may include nodes located on different concentric 3D mesh layers.

[0228] As a non-limiting example, the alignment procedure 380 may be implemented by determining whether the nodes in the initial mesh are aligned with multiple elements p e After determining the anchor point, the registration procedure 380 may determine that the feature (i.e., biomarker) value at that node corresponds to p e Correspondence rules between multiple elements in a given domain representation can be generated by specifying that a weighted average of the biomarkers of

[0229] Additionally or alternatively, for elements in a given domain representation that do not directly correspond to nodes on an initial 3D mesh comprising multiple concentric 3D mesh layers, the registration procedure 380 may use distance to weight biomarker values ​​in the correspondence rule.

[0230] The alignment procedure 380 outputs a separate correspondence rule for each domain representation.

[0231] Parametric Mesh Encoding Procedure The parametric mesh encoding procedure 390 is configured to, among other things, (i) receive an initial 3D mesh; (ii) receive a multi-domain representation from the multi-domain data acquisition procedure 350; (iii) receive correspondence rules from the alignment procedure 380; (iv) use the correspondence rules to determine, for at least one given region, individual feature sets from individual biomarkers in the multi-domain representation; and (v) assign the feature sets to the at least one given region of the initial 3D mesh to obtain a 3D parametric mesh, wherein each node of the at least one region of the parametric mesh is associated with an individual plurality of feature channels that include at least the feature set.

[0232] The goal of the parametric mesh encoding procedure 390 is to use the initial 3D mesh to generate a 3D parametric mesh that is a single representation of the patient's body anatomy, including all biomarker data from multiple domain representations. It will be appreciated that the biomarker data may be temporally encoded onto the initial mesh. The 3D parametric mesh may then be provided for display on a user interface and used to render, display, and interact with visual representations of the patient-specific data, including displaying different views in 2D, 3D, and 4D, and displaying the multi-domain biomarker data encoded within the nodes of the 3D parametric mesh.

[0233] Determining characteristics The parametric mesh encoding procedure 390 is configured to determine a distinct set of feature values ​​from the given domain representation for each region of the given domain representation that has a corresponding region in the mesh using a distinct correspondence rule.

[0234] The set of distinct features corresponds to at least a portion of the biomarkers in a given domain representation. It will be appreciated that the set of distinct features extracted may vary depending on the region, concentric mesh layer, and domain representation. The features may include all biomarkers from a given domain representation, or may include only biomarkers of interest from a given domain representation (which may be predetermined by an operator).

[0235] The parametric mesh encoding procedure 390 uses individual correspondence rules to convert biomarker values ​​into nodal feature values ​​that are assigned to each node in the corresponding region in the initial 3D mesh.

[0236] The parametric mesh encoding procedure 390 is configured to encode structural information from a set of segmented images including multiple anatomical segments as features within a mesh. Structural information (i.e., locations) of segments of a given patient's anatomy are encoded into nodes of an initial 3D mesh, resulting in a parametric 3D mesh that can be used to render 2D and / or 3D representations of the physical structure of the corresponding anatomy displayed in the given patient's image set. The parametric mesh encoding procedure 500 may encode structural information, such as the location of the anatomical structure's centerline, boundaries, and locations of substructures (e.g., locations of the lumen, outer wall, thrombus, and calcification in a segmented aorta), into the parametric 3D mesh. The structural information encoded as features within the nodes of the 3D parametric mesh is used to render the 3D parametric mesh on a user interface. It is understood that structural information (if available) can be encoded over time, so that changes in the parametric 3D mesh over time can also be visually represented (e.g., the location of the lumen, outer wall, thrombus, and calcification at various times during the cardiac cycle).

[0237] The parametric mesh encoding procedure 390 is configured to encode the biomarkers of corresponding elements in the structural mechanics representation to associated nodes of the mesh using distinct correspondence rules. Each structural mechanics representation biomarker is encoded as a separate feature in multiple feature channels of the corresponding node. It will be appreciated that feature positions within the multiple feature channels may be reserved for the structural mechanics biomarkers, e.g., the structural mechanics biomarkers may be encoded in channels 10-20 of each node.

[0238] The parametric mesh encoding procedure 390 is configured to encode biomarkers of corresponding elements of the fluid dynamics representation into associated nodes of the mesh using distinct correspondence rules. Each of the fluid dynamics representation biomarkers is encoded as a separate feature in multiple feature channels of the corresponding node. It will be appreciated that feature positions within the multiple feature channels may be reserved for the fluid dynamics biomarkers, e.g., structural dynamics biomarkers may be encoded in channels 21-30 of each node.

[0239] The parametric mesh encoding procedure 390 is configured to encode the biomarkers of corresponding elements in the descriptive variable representation into associated nodes of the mesh using distinct correspondence rules. Each descriptive variable representation biomarker is encoded as a separate feature in multiple feature channels of the corresponding node. It will be appreciated that feature positions within the multiple feature channels may be reserved for structural dynamics biomarkers, e.g., structural dynamics biomarkers may be encoded in channels 31-50 of each node.

[0240] It will be appreciated that all biomarker data of interest for a given region of interest in the anatomy can thus be readily stored and retrieved using a single coordinate system of the parametric 3D mesh. Thus, for a given node corresponding to a region in the anatomy, e.g., a peripheral point on the ascending aorta, biomarker data from all modalities and physical representations associated with that given node can be encoded as features such as, for example, location, time, pixel intensity, strain values ​​including maximum principal strain, minimum principal strain, circumferential strain, and longitudinal strain, deformation values, fluid dynamics data, etc.

[0241] The parametric mesh encoding procedure 390 outputs a 3D parametric mesh composed of multiple concentric 3D mesh layers, each having a predetermined number of nodes, each containing multiple feature channels. The 3D parametric mesh is a patient-specific representation of the patient's anatomy, allowing for intuitive and systematic reporting of information from multiple domains on anatomically relevant maps extracted from the patient's original vascular scan.

[0242] Although the registration procedure 380 and the parametric mesh encoding procedure 390 have been described as separate procedures, it will be understood that such description is for illustrative purposes only, and that the registration procedure 380 and the parametric mesh encoding procedure 500 may be combined.

[0243] In one or more embodiments, the parametric mesh may be stored in a non-transitory storage medium or database 235 of the server 230 .

[0244] In one or more embodiments, the parametric mesh may be output and transmitted.

[0245] As a non-limiting example, the parametric mesh may be transmitted to a workstation computer 215 for display.

[0246] The parametric mesh may be displayed using any suitable 2D or 3D rendering technique known in the art and may be interacted with to visualize data from multiple feature channels.

[0247] The parametric mesh can thus provide a database containing all structural, functional, and descriptive data for the anatomical structures of interest for a given patient, so that the data encoded in the form of features at each node location can be rapidly retrieved and displayed for analysis.

[0248] The parametric mesh generation procedure 300 is repeated for multiple patients to obtain separate parametric meshes that each encode separate biomarkers for a separate patient.

[0249] The set of parametric meshes may be used to train various ML models, and a given parametric mesh from the set of parametric meshes may be associated with an individual patient.

[0250] It is understood that because all ML models rely on the same data types, weights can be shared or retrained very minimally when new information domains are introduced. Modular modeling makes it possible to retrain new architectures or for new tasks by utilizing fewer weights (parameters) and requiring less retraining of these weights. Consequently, this makes it easier to obtain generalizable models starting with a smaller number of vascular scans.

[0251] FIG. 4 shows a non-limiting example of a perspective view of a rendering of a first parametric mesh 400 taken from the anterior left side of the aorta including the iliac arteries, in accordance with one or more non-limiting embodiments of the present technology.

[0252] FIG. 5A shows a perspective view of a rendering of the first parametric mesh 400 of FIG. 4 with the top portion removed according to line 11, revealing multiple concentric 3D mesh layers 440 beneath the first parametric mesh 400.

[0253] FIG. 5B shows an enlarged detailed view of the multiple concentric 3D mesh layers 440 of FIG. 5A, including a selected node 454 and its multiple feature channels 460.

[0254] In this illustrated example, the plurality of concentric 3D mesh layers 440 includes ten layers (not all of which are numbered): a first concentric 3D mesh layer 442, a second concentric 3D mesh layer 444, a third concentric 3D mesh layer 446, ..., a ninth concentric 3D mesh layer 448, and a tenth concentric 3D mesh layer 450.

[0255] The first concentric 3D mesh layer 442 is the innermost layer and represents the core (not numbered) of the first parametric mesh 400 .

[0256] The tenth concentric 3D mesh layer 450 is the outermost layer and represents the outer wall of the aorta.

[0257] As a non-limiting example, the first concentric 3D mesh layer 442 can be used to encode features such as the diameter and curvature of the aortic centerline, while the tenth layer 450 can be used to encode wall distortion, shape, and the presence of calcification. The intermediate concentric layers, i.e., the second concentric 3D mesh layer 444, the third concentric 3D mesh layer 446, ..., the ninth concentric 3D mesh layer 448, can be used to encode the presence of thrombus, flow patterns, image pixel color, etc., derived from biomarkers in multiple domains (e.g., different imaging modalities, computational fluid dynamics, etc.).

[0258] Although the tenth concentric 3D mesh layer 450 is shown as the outermost layer of the first parametric mesh 400, it will be understood that in one or more other embodiments, the first parametric mesh 400 may include one or more additional layers located outside the represented anatomical structure (e.g., outside the outer wall of the aorta). Such additional layers may or may not correspond to other anatomical structures and may be used to encode additional information.

[0259] FIG. 6 shows a top or axial plan view of a rendering of the first parametric mesh 400 of FIG. 5B with the top portion removed according to line 11.

[0260] 5B and 6, individual nodes 454 of the ninth concentric 3D mesh layer 448 of the first parametric mesh 400 are associated with respective node coordinates 456 represented as (m,n,p), where m is the circumferential coordinate, n is the longitudinal coordinate, and p is the time frame coordinate. Each node 454 of the mesh 400 has multiple feature channels 460 encoding features derived from different biomarkers. The multiple feature channels 460 may include, for example, structural features 472, hydrodynamic features 476, and variable descriptive features 478.

[0261] FIG. 7 shows a perspective view of a rendering of a second parametric mesh 700 taken from the anterior left side of the aorta and iliac arteries, in accordance with one or more non-limiting embodiments of the present technology.

[0262] The second parametric mesh 700 includes an aortic mesh 720 , a left iliac artery mesh 724 , and a right iliac artery mesh 726 .

[0263] In the illustrated second parametric mesh 700, only the outermost or surface concentric 3D mesh layers are shown, with each axial layer represented by an elliptical shape for each discrete axial value (i.e., each longitudinal value). A centerline 730 (corresponding to a node in the first 3D layer of the aortic mesh 720) extends within the aortic mesh 720 and bifurcates into a left iliac centerline 734 (corresponding to a node in the first 3D layer of the left iliac mesh 724) extending within the left iliac mesh 724, and a right iliac centerline 736 (corresponding to a node in the first 3D mesh layer of the right iliac mesh 726) within the right iliac mesh 726.

[0264] A second aortic line 740 extends within the aortic mesh 720 and is defined by nodes having the same circumferential coordinate m but different longitudinal coordinates n (i.e., located on different axial layers). The second aortic line 740 bifurcates into a second left iliac line 744 and a second right iliac line 746. Like the nodes in the second aortic line 740, the nodes in each of the second left iliac line 744 and the second right iliac line 746 have the same circumferential coordinate m in their respective arrangements but different longitudinal coordinates n in their respective arrangements (i.e., located on different axial layers).

[0265] FIG. 8 shows a schematic diagram of a first user interface 800 displaying a third parametric mesh 805, a corresponding third parametric mesh array 850, and visual renderings of user interface elements in the form of a layer slider 870, a domain slider 880, and a substructure slider 890.

[0266] A third parametric mesh 805 is 3D rendered on the left side of the user interface 800. The third parametric mesh 805 includes an aortic mesh 810, a left iliac artery mesh 830, and a right iliac artery mesh 840. It will be appreciated that the visual representation of the third parametric mesh 805 is generated from the structural features encoded in the nodes of the third parametric mesh 805, and visually represents the particular anatomy of a given patient from which multi-domain information was extracted.

[0267] A third parametric mesh array 850 is displayed in the upper right portion of the user interface 800. The third parametric mesh array 850 includes an aortic mesh array 852, a left iliac artery array 856, and a right iliac artery array 858.

[0268] The third parametric mesh array 850 represents the third parametric mesh 805 unwrapped relative to the centerline (or vertical axis), with the columns of the outer layer array 850 corresponding to the vertical node positions (i.e., positions along the vertical axis) on the third parametric mesh 805 and the rows of the third parametric mesh array 850 corresponding to the circumferential node positions (i.e., positions along the circumference). Each array element in the third parametric mesh array 850 corresponds to a distinct node on a visual rendering of the third parametric mesh 805.

[0269] In the illustrated non-limiting example, it can be seen that necks 820 in aortic mesh 810 are represented by neck array 854 in aortic mesh array 852, left iliac artery mesh 830 is represented by left iliac artery array 856, and right iliac artery mesh 840 is represented by right iliac artery array 858.

[0270] Each node in the outer layer of the third parametric mesh 805 can be accessed in the third parametric mesh array 850 using the same coordinates (M,N) as above, where M is the circumferential node coordinate and N is the longitudinal node coordinate. It will be appreciated that each concentric mesh layer of the third parametric mesh 805 can be represented as a separate array of the same size as the third parametric mesh array 850, and each time frame (i.e., corresponding to the parametric mesh 805 at a different point in time) can be represented as a respective array of the same size.

[0271] The layer slider 870 allows for the selection and display of different concentric 3D mesh layers of the third parametric mesh 805, including the innermost core layer, wall layers, and outer layers, as well as layers located in between. By selecting a layer using the layer slider 870, structural information encoded in the nodes of that layer may be processed to render a graphical representation of the selected mesh layer.

[0272] The graphic rendering displayed on the left changes depending on the layer selected in the layer slider 870, but the size and structure of the corresponding third parametric mesh array 850 remains the same, i.e., it contains the same number of nodes or elements.

[0273] The domain slider 880 allows for the selection and display of different domains encoded in the third parametric mesh 805, including, by way of non-limiting examples, strain, computational fluid dynamics (CFD), and calcification. By selecting a domain using the domain slider 880, biomarkers encoded as features at the nodes may be processed to render a graphical representation of the selected domain. The user interface 800 also includes a substructure slider 890 in the form of a neck parameter slider that allows for the display of different features specific to the neck in the aorta.

[0274] Although not shown in FIG. 8, the user may also use other interface elements to select different types of rendering and projection.

[0275] FIG. 9A shows a schematic diagram of the second user interface 900 showing a visual representation of a fourth parametric mesh 905 of the aorta including the iliac arteries, where the concentric mesh layer 915 between the lumen and wall is highlighted after being selected with the layer slider 930, and the corresponding layer array 920 shows grayscale pixel intensities after the "grayscale" domain is selected with the domain slider 940.

[0276] FIG. 9B shows a schematic diagram of the second user interface of FIG. 9A, in which the core concentric mesh layer 965 selected with the layer slider 930 is highlighted, and the corresponding layer array 970 displays the grayscale pixel intensities of the core concentric mesh layer 965 with the “grayscale” domain selected with the domain slider 940.

[0277] Having described the parametric mesh generation procedure 300 with reference to Figure 3 and various examples of parametric meshes with reference to Figures 4-9B, reference is now made to Figures 10A and 10B, which show a flowchart of a method 1000 for generating a parametric mesh in accordance with one or more non-limiting embodiments of the present technology.

[0278] It will be appreciated that the procedure(s) of the parametric mesh generation procedure 300 may be integrated into the method 1000 .

[0279] Instructions In one or more embodiments, server 230 comprises at least one processor, such as processor 110 and / or GPU 111, operably connected to a non-transitory computer-readable storage medium, such as solid-state drive 120 and / or random access memory 130, that stores computer-readable instructions. The at least one processor is configured to or operable to perform method 1000 upon execution of the computer-readable instructions.

[0280] The method 1000 begins at process step 1002 .

[0281] According to processing step 1002, at least one processor receives an image set of a given patient acquired by a medical imaging device, the image set including at least one image of at least some of the anatomical structures within the given patient's body.

[0282] In one or more embodiments, the image set is acquired by a medical imaging device 210 .

[0283] In one or more other embodiments, the image set may include multiple images in the form of an image stack. In one or more alternative embodiments, the image set includes multiple images in the form of a multiphase stack.

[0284] In one or more embodiments, the anatomical structure includes the aorta of a given patient. Additionally, the anatomical structure may include the iliac arteries of a given patient.

[0285] According to process step 1004, at least one processor segments the image set to obtain a plurality of anatomical segments of at least some of the internal anatomy of a given patient.

[0286] In one or more embodiments, at least one processor obtains a plurality of anatomical segments or segmented tissues using manual and / or automatic segmentation methods.

[0287] In one or more embodiments, at least one processor uses a set of trained segmentation models 260 to segment a set of images to obtain a plurality of segments.

[0288] As a non-limiting example, the set of trained segmentation models 260 is trained to segment an aortic region that includes one or more of the aorta and iliac arteries, and the segmented aortic region includes an ROI that includes the lumen, the aortic wall, the ILT (if present), and calcification (if present).

[0289] In one or more embodiments, processing steps 1002 and 1004 may be replaced by a single processing step in which a processor receives multiple anatomical segments from another processor.

[0290] According to processing step 1006, at least one processor receives an initial 3D mesh for representing an anatomical structure, the initial 3D mesh including a plurality of concentric 3D mesh layers, each of the plurality of concentric 3D mesh layers including the same predetermined number of nodes.

[0291] In one or more embodiments, each node has at least one distinct time coordinate that allows the node and the mesh to be represented in time.

[0292] The processor initializes the mesh according to a set of mesh parameters that specify at least the geometry and number of nodes of the mesh, where each predetermined region in the mesh may be, for example, part of one or more anatomical structures of interest and may correspond to at least a portion of a segmented anatomical segment obtained by segmenting the image set in processing step 1004.

[0293] In one or more embodiments, process step 1006 may be performed before process steps 1002 and 1004 .

[0294] According to processing step 1008, at least one processor determines at least one region in the mesh that corresponds to a given anatomical segment of the plurality of anatomical segments to obtain a correspondence rule between the at least one region in the mesh and the given anatomical segment.

[0295] In one or more embodiments, the correspondence rule represents a functional mapping element from a region within a given anatomical segment of the plurality of anatomical segments to a corresponding region of a node in the initial mesh.

[0296] According to processing step 1010, at least one processor uses a correspondence rule to encode at least one node set of the 3D mesh with a distinct set of features from at least one distinct anatomical segment to obtain a 3D parametric mesh, wherein each node of the at least one node set in the 3D parametric mesh is associated with a distinct plurality of feature channels comprising the distinct set of features.

[0297] In one or more embodiments, processing step 1010 includes determining a set of distinct features from biomarkers in at least one distinct anatomical segment using distinct correspondence rules, and assigning a distinct set of features from the at least one distinct anatomical segment to each of the at least one distinct set of nodes.

[0298] In one or more embodiments, at least one processor uses correspondence rules to extract biomarkers, such as locations of anatomical segments and pixel intensities from the anatomical segments.

[0299] It will be appreciated that because the discretized geometry of the initial mesh may differ from the discretized geometry of the multiple segments in the image, the correspondence rule may specify which regions and cell locations within the multiple segments correspond to nodes in the mesh and how the cell values ​​should be reported to the initial 3D mesh. As a non-limiting example, each node may correspond to an area of ​​four pixels in the segmented image, and therefore the correspondence rule may specify that the biomarker values ​​of the four pixels should be averaged to obtain a feature. Thus, the processor determines features based on the biomarker values ​​of each node based on the correspondence rule and populates each node with the determined features.

[0300] At least one processor encodes or populates a set of features of at least one region within the segment into nodes of the mesh. The set of features may include, for example, location and pixel intensity. Thus, the 3D parametric mesh provides a discretized 2D or 3D representation of regions within the anatomical segment that are encoded as nodal features. In one or more embodiments, temporal information may be encoded as features at the nodes of the parametric mesh, allowing for visualization of the evolution of the features over time. The nodal features may be used to generate a visual representation of the 3D parametric mesh and may be displayed in one or more arrays corresponding to the 3D parametric mesh.

[0301] It will be appreciated that processing steps 1008-1010 may be repeated for other regions and segments such that all necessary information about the anatomical structure of interest is encoded in the 3D parametric mesh.

[0302] According to processing step 1012, at least one processor receives a domain representation including biomarkers associated with internal anatomical structures of a given patient.

[0303] In one or more embodiments, the processor receives at least one of a structural mechanics representation including structural mechanics biomarkers, a fluid mechanics representation including fluid mechanics biomarkers, and a descriptive variable representation including variable descriptive biomarkers.

[0304] Each domain representation may have a different data format and / or data density and / or resolution.

[0305] In one or more embodiments, the given domain representation may be a mesh, such as a polygonal mesh. A polygonal mesh includes vertices, edges, and faces. The faces may include one of triangles (triangle mesh), quadrilaterals (quads), convex polygons (n-gons), concave polygons, and polygons with holes. The mesh may be a 2D or 3D mesh, with or without a temporal component.

[0306] In one or more other embodiments, the given domain representation may be a 2D or 3D image.

[0307] In one or more embodiments, the processor uses a registration technique to determine another correspondence rule.

[0308] The individual correspondence rules may include a functional mapping element from a given domain representation to a corresponding element in the initial mesh, thereby allowing biomarker data associated with elements of the given domain representation to be mapped as features on the initial mesh. In other words, the individual correspondence rules are functions that describe the mapping from the individual coordinate system of the domain representation to the coordinate system of the initial mesh.

[0309] According to processing step 1014, the processor determines another region in the domain representation and another region in the parametric mesh that corresponds to another given anatomical segment to obtain another correspondence rule.

[0310] It will be appreciated that the other region may be the same region as in process step 1008 or a different region.

[0311] According to processing step 1016, the processor uses another individual correspondence rule, at least one other individual set of nodes in the 3D parametric mesh with another feature set based on the individual biomarkers, and each node of the at least one other individual set of nodes in the 3D parametric mesh is associated with multiple individual feature channels comprising the other feature set.

[0312] In one or more embodiments, the number of features or features represented in the feature channels of each node in the 3D parametric mesh may vary.

[0313] In one or more embodiments, to perform processing step 1016, the processor determines a different set of features from the biomarkers associated with the given anatomical segment in the domain representation based on a different correspondence rule and assigns the different set of features to each node of at least one other distinct set of nodes in the 3D parametric mesh.

[0314] It will be appreciated that processing steps 1014-1016 may be performed multiple times, each for a different domain representation containing a distinct biomarker.

[0315] The method 1000 then ends.

[0316] One or more embodiments of the present technology provide an anatomically relevant meshing strategy to produce homogenized data across multiple modalities and scans. Parametric meshes generated using the present disclosure allow data from all data types, ranging from shell and solid meshes to array-like data including pixel-specific data, to be stored in stackable layers that rely on a single type of data encoding and are easy to interpret and utilize for training more compact machine learning-based models.

[0317] One or more embodiments of the present method and system convert multiple vessel-specific data types into multi-channel, anatomically relevant, stackable images that can be used to train diagnostic and prognostic artificial intelligence-based models, as well as provide organized, intuitive multi-domain reports to medical professionals.

[0318] One or more embodiments of the present technology enable modular modeling for diagnostic and prognostic purposes, leveraging each available domain of information. Because all models rely on the same data types, weights can be shared or retrained very minimally when new domains of information are introduced. Modular modeling makes it possible to retrain new architectures or for new tasks by utilizing fewer weights (parameters) and requiring less retraining of these weights. Consequently, this makes it easier to obtain generalizable models starting with a smaller number of vascular scans.

[0319] In some cases, what are believed to be useful examples of modifications to the technology may also be described. This is done merely to aid in understanding and, again, does not define the scope or delimit the technology. These modifications are not an exhaustive list, and one of ordinary skill in the art may nonetheless make other modifications while remaining within the scope of the technology. Furthermore, if examples of modifications are not described, it should not be construed that modifications are not possible and / or that what is described is the only way to implement that element of the technology.

[0320] Modifications and improvements to the above-described embodiments of the technology may become apparent to those skilled in the art. The foregoing description is intended to be illustrative, not limiting.

Claims

1. 1. A method for generating a 3D parametric mesh of an anatomical structure and storing multi-domain data therein, the method being executed by at least one processor and comprising: receiving a plurality of anatomical segments of at least some of the anatomical structures within a given patient's body obtained from a segmentation of an image set acquired by a medical imaging device, the image set including at least one image of at least some of the anatomical structures within the given patient's body; receiving a 3D mesh representing the anatomical structure, the 3D mesh comprising: a plurality of concentric 3D mesh layers, each of the plurality of concentric 3D mesh layers having the same predetermined number of nodes; receiving, determining at least one distinct set of nodes in the 3D mesh corresponding to at least one distinct anatomical segment of the plurality of anatomical segments to obtain a distinct correspondence rule therebetween; encoding the at least one distinct set of nodes of the 3D mesh with a distinct set of features from the at least one distinct anatomical segment using the correspondence rule to obtain a 3D parametric mesh, wherein each node of the at least one distinct set of nodes in the 3D parametric mesh is associated with a distinct plurality of feature channels comprising the distinct set of features; A method comprising:

2. The method of claim 1 , wherein at least a subset of nodes of the at least one set of nodes are located on different concentric 3D mesh layers.

3. encoding the at least one set of nodes of the 3D mesh with the respective set of features from the at least one respective anatomical segment using the correspondence rule to obtain the 3D parametric mesh, determining a distinct set of features from biomarkers in the at least one distinct anatomical segment using the distinct correspondence rules; assigning to each of the at least one distinct set of nodes the distinct set of features from the at least one distinct anatomical segment; 3. The method of claim 1 or 2, comprising:

4. receiving a domain representation including individual biomarkers associated with the anatomical structures within the body of the given patient; determining at least one other distinct set of nodes in the 3D parametric mesh that corresponds to at least one other region in the domain representation to obtain another distinct correspondence rule, wherein at least a subset of the other distinct second set of nodes is located on a different concentric 3D mesh layer; encoding the at least one other distinct set of nodes in the 3D parametric mesh with another feature set based on the distinct biomarkers using the other distinct correspondence rule, wherein each node of the at least one other distinct set of nodes in the 3D parametric mesh is associated with a distinct plurality of feature channels comprising the other feature set; The method of any one of claims 1 to 3, further comprising:

5. The method of any one of claims 1 to 4, wherein each individual node is further associated with at least one time frame for representing said 3D parametric mesh in time.

6. 6. The method of claim 1, wherein each distinct concentric 3D mesh layer is represented as a distinct multidimensional array, and the position of a given node on the distinct 3D mesh layer corresponds to the position of the given node in the distinct multidimensional array.

7. 7. The method of claim 6, wherein the plurality of feature channels for each node of the respective 3D mesh layer are represented as a respective node array, each cell of the respective node array corresponding to a respective feature channel of the plurality of feature channels.

8. 8. The method of claim 2, wherein the domain representation includes another mesh that is different from the 3D parametric mesh, and the other individual correspondence rule includes determining a mapping between nodes in the other mesh and nodes in the 3D parametric mesh.

9. 9. The method of claim 8, wherein the another mesh comprises one of a polygonal mesh, the polygonal mesh comprising one of a triangular mesh, a quadrilateral mesh, a convex polygonal mesh, a concave polygonal mesh, and a polygonal mesh with holes.

10. the domain representation includes a structural mechanics representation; The method of any one of claims 2 to 9, wherein the individual biomarkers include structural mechanics biomarkers, the structural mechanics biomarkers including at least one of a pressure value, a strain value, and a deformation value.

11. the domain representation includes a fluid dynamics representation; The method of any one of claims 2 to 9, wherein the individual biomarkers include at least one of a blood flow value and a shear stress value.

12. the domain representation includes a descriptive variable representation; The method of any one of claims 2 to 9, wherein the individual biomarkers comprise at least one of geometric and image data values.

13. The method of any one of claims 1 to 12, wherein the anatomical structure comprises the aorta of the given patient.

14. The method of claim 13 , wherein the plurality of anatomical segments includes a lumen and an aortic wall.

15. 1. A system for generating a 3D parametric mesh of an anatomical structure and storing multi-domain data therein, comprising: at least one processor; a non-transitory storage medium operably connected to the at least one processor, the non-transitory storage medium storing computer-readable instructions; and Equipped with The at least one processor, when executing the computer-readable instructions, receiving a plurality of anatomical segments of at least some of the anatomical structures within a given patient's body obtained from segmentation of an image set acquired by a medical imaging device, the image set including at least one image of at least some of the anatomical structures within the given patient's body; receiving a 3D mesh representing the anatomical structure, the 3D mesh comprising: receiving a plurality of concentric 3D mesh layers, each of the plurality of concentric 3D mesh layers including the same predetermined number of nodes; determining at least one distinct set of nodes in the 3D mesh corresponding to at least one distinct anatomical segment of the plurality of anatomical segments to obtain a distinct correspondence rule therebetween; encoding the at least one distinct set of nodes of the 3D mesh with a distinct set of features from the at least one distinct anatomical segment using the correspondence rule to obtain a 3D parametric mesh, wherein each node of the at least one distinct set of nodes in the 3D parametric mesh is associated with a distinct plurality of feature channels comprising the distinct set of features; A system that is configured to:

16. 16. The system of claim 15, wherein at least a subset of nodes of the at least one set of nodes are located on different concentric 3D mesh layers.

17. encoding the at least one set of nodes of the 3D mesh with the respective set of features from the at least one respective anatomical segment using the correspondence rule to obtain the 3D parametric mesh, determining a distinct set of features from biomarkers in the at least one distinct anatomical segment using the distinct correspondence rules; assigning to each of the at least one distinct set of nodes the distinct set of features from the at least one distinct anatomical segment; 17. The system of claim 15 or 16, comprising:

18. The at least one processor further comprises: receiving a domain representation including individual biomarkers associated with the anatomical structures within the body of the given patient; determining at least one other distinct set of nodes in the 3D parametric mesh that corresponds to at least one other region in the domain representation to obtain another distinct correspondence rule, wherein at least a subset of the other distinct second set of nodes is located on a different concentric 3D mesh layer; encoding the at least one other distinct set of nodes in the 3D parametric mesh with another feature set based on the distinct biomarkers using the other distinct correspondence rule, wherein each node of the at least one other distinct set of nodes in the 3D parametric mesh is associated with a distinct plurality of feature channels comprising the other feature set; The system according to any one of claims 15 to 17, configured to:

19. The system of any one of claims 15 to 18, wherein each individual node is further associated with at least one time frame for representing said 3D parametric mesh in time.

20. 20. The system of claim 15, wherein each distinct concentric 3D mesh layer is represented as a distinct multidimensional array, and the position of a given node on the distinct 3D mesh layer corresponds to the position of the given node in the distinct multidimensional array.

21. 21. The system of claim 20, wherein the plurality of feature channels for each node of the respective 3D mesh layer are represented as a respective node array, each cell of the respective node array corresponding to a respective feature channel of the plurality of feature channels.

22. 22. The system of claim 16, wherein the domain representation includes another mesh that is different from the 3D parametric mesh, and the other individual correspondence rule includes determining a mapping between nodes in the other mesh and nodes in the 3D parametric mesh.

23. 23. The system of claim 22, wherein the another mesh comprises one of a polygonal mesh, the polygonal mesh comprising one of a triangular mesh, a quadrilateral mesh, a convex polygonal mesh, a concave polygonal mesh, and a polygonal mesh with holes.

24. the domain representation includes a structural mechanics representation; The system of any one of claims 16 to 23, wherein the individual biomarkers include structural mechanics biomarkers, the structural mechanics biomarkers including at least one of a pressure value, a strain value, and a deformation value.

25. the domain representation includes a fluid dynamics representation; The system of any one of claims 16 to 23, wherein the individual biomarkers include at least one of a blood flow value and a shear stress value.

26. the domain representation includes a descriptive variable representation; The system of any one of claims 16 to 23, wherein the individual biomarkers include at least one of geometric and image data values.

27. The system of any one of claims 15 to 26, wherein the anatomical structure includes the aorta of the given patient.

28. 28. The system of claim 27, wherein the plurality of anatomical segments includes a lumen and an aortic wall.

Images