Implant generation system

Abstract

Methods and systems for generating implants for use in cosmetic or reconstructive surgery are disclosed. A three-dimensional (3D) model of a bodily portion of the subject is obtained 102. The model co

Description

Technical Field [oooi] The present application relates to apparatus, systems and method(s) for generating one or more implants from a 3D model of a subject. 5 Background

[0002] CN 108904099 A describes a personalized implant design method based on bone and skin deformation simulation. US 2015 / 0198943 Ai describes a method of designing, presenting, generating, and fabricating custom implants by obtaining a 3D image of a site to receive an implant, simulating volumetric changes of the site, 10 generating a virtual 3D implant that effects the volumetric changes of the site, and fabricating a real 3D implant that includes the volumetric changes. US 2007 / 0276501 Ai describes generating custom spinal implants by programmatically analyzing a patient's image data to electronically obtain shapes and dimensions of relevant anatomical features of a target region of the patient’s spine; and fabricating a patient- 15 specific replacement spinal implant for the patient using the analyzed patient image data. US 2022 / 0202560 Ai describes a multi-modal imaging methodology for creating patient-specific three-dimensional images of a breast volume, for use in the production of breast implants. In the field of cosmetic and reconstructive craniofacial surgery facial implants are placed on the skull and change the morphology and appearance of 20 the face of a subject (e.g. a patient). Facial implants include implants such as zygomatic, maxillaiy, forehead, orbital and mandibular implants. Conventional design and manufacture of facial implants is still highly dependent on the skill of the surgeon to shape the facial implant for a subject rather than being precisely engineered for that subject. Most facial implants are designed to reconstruct or enhance facial form of the 25 subject, but it is currently not possible to accurately predict or even present to the subject the change in facial shape a specific implant will induce. Neither do existing implant design techniques nor processes provide the subject with a visualised or realistic prediction of their post-operative facial appearance.

[0003] Existing facial implant design techniques or processes rely on computer-30 aided-design (CAD) based 3-D design programs. Currently implants are designed using an inside out process; i.e. determine the shape of the implant based on anatomical norms and infer what changes in facial shape of the subject this will induce. Implants can be designed and then shaped to restore normal anatomical shape to the bone of a subject. Whilst restoring normal bony anatomy, this technique does not take into account any overlying soft tissue anomalies, which makes the final change in facial 5 appearance of a subject very unpredictable. As well, this usually leads to adjustments being made to the facial implant during surgery, which may also lead to unpredictable results, and more repeat visits.

[0004] Similarly, other cosmetic and reconstructive surgery performed on other parts of the body of a subject (e.g. chest, calves, legs, arms) also involve designing 10 implants to change or modify the appearance of the underlying bone / muscle of the body part of the subject. Again, there are no standard techniques that may be applied 27 07 23 to enable either the subject or surgeon to predict or even present to the subject the change in shape of the body part that a specific implant will induce. Although the subject may be directed to look at other cases and / or the experience of the surgeon to describe the impact an implant may have, it is difficult for the subject to understand what to expect as a result of the surgery. As well, given most implants are generic, last minute adjustments maybe required by the surgeon to the implant during surgery. This can also lead to unpredictable results, and more repeat corrective visits.

[0005] There is a desire to improve the accuracy, efficiency and predictability of the implant design and manufacture process. Summary

[0006] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter; variants and alternative features which facilitate the working of the invention and / or serve to achieve a substantially similar technical effect should be considered as falling into the scope of the invention disclosed herein.

[0007] According to a first aspect, there is described a computer-implemented method of generating one or more implants for a subject for use in cosmetic or reconstructive surgery, the method comprising: obtaining a three-dimensional (3D) model of a bodily portion of the subject, the 3D model comprising: 3D photogrammetry data representative of the bodily portion of the subject and defining a desired external appearance of the bodily portion of the subject; and a 3D medical model representative of the bodily portion of the subject, the 3D medical model comprising a first 3D structure and a second 3D structure underlying the first 3D structure in relation to the bodily portion of the subject; determining one or more volumes generated between the 3D photogrammetry data of the desired bodily portion of the subject and the second 3D structure taking into account the structural properties of at least the first 3D structure; and outputting data representative of the determined one or more volumes for controlling the manufacture of one or more corresponding implants for the subject. [ooo8] In some example embodiments, determining the one or more volumes further comprising: adjusting the first and second 3D structures taking into account the structural properties of at least the first 3D structure until the outer surface of the first 3D structure substantially coincides with the desired 3D photogrammetry data; and calculating one or more volumes generated between the adjusted second 3D structure and the original second 3D structure based on determining one or more volume shapes resulting from differences between the adjusted second 3D structure and the original second 3D structure.

[0009] In some example embodiments, adjusting the first and second 3D structures taking into account at least the properties of the first 3D structure further comprising iteratively increasing or decreasing regions of the outer surface of the second 3D structure whilst modelling the structural properties of the first 3D structure until the outer surface of the first 3D structure substantially coincides with the desired 3D photogrammetry data.

[0010] In some example embodiments, iteratively adjusting the second 3D structure further comprising determining the extent of the first 3D structure based on modelling the structural properties of the first 3D structure in relation to adjustments to the outer surface of the second 3D structure until the outer surface of the first 3D structure substantially coincides with the desired 3D photogrammetry data.

[0011] In some example embodiments, taking into account the structural properties of the first and second 3D structures further comprising performing finite element analysis on the first and second 3D structures to adjust the outer surfaces of the first and second 3D structures based on the structural properties of the first 3D structure and until the outer surface of the first 3D structure substantially coincides with the desired 3D photogrammetry data.

[0012] In some example embodiments, the first 3D structure comprises a first 3D structural mesh representing the outer surface of the first 3D structure, the second 3D structure comprises a second 3D structural mesh representing the outer surface of the second 3D structure, and the 3D photogrammetry data of the desired bodily portion comprises a desired 3D photogrammetry mesh, and determining the one or more volumes further comprising: adjusting the first and second 3D structural meshes taking into account at least the properties of the first 3D structure until the first 3D structural mesh substantially coincides with the desired 3D photogrammetry mesh; and calculating any volumes generated between the adjusted second 3D structural mesh and the original second 3D structural mesh.

[0013] In some example embodiments, adjusting the first and second 3D structural meshes taking into account at least the properties of the first 3D structure further comprising iteratively increasing outwardly or decreasing inwardly the second 3D structural mesh whilst modelling the structural properties of the first 3D structure until the first 3D structural mesh substantially coincides with the desired 3D photogrammetry mesh.

[0014] In some example embodiments, taking into account the structure of the first and second 3D structures further comprising performing finite element analysis on the first and second 3D structures to adjust the boundaries of the first 3D structural mesh and second 3D structural mesh based on modelling the structural properties of the first 3D structure and until the first 3D structural mesh substantially coincides with the desired 3D photogrammetry mesh.

[0015] In some example embodiments, taking into account the structural properties of the first and second 3D structures further comprising one or more from the group of: performing finite element analysis on the first and second 3D structures to adjust the outer surfaces of the first and second 3D structures based on the structural properties of the first 3D structure and until the outer surface of the first 3D structure substantially coincides with the desired 3D photogrammetry data; performing massspring analysis on the first and second 3D structures to adjust the outer surfaces of the first and second 3D structures based on the structural properties of the first 3D structure and until the outer surface of the first 3D structure substantially coincides with the desired 3D photogrammetry data; and performing volumetric analysis on the first and second 3D structures to deform the outer surfaces of the first and second 3D structures based adjustments required in relation to the desired 3D photogrammetry data.

[0016] In some example embodiments, calculating the one or more volumes generated between the adjusted second 3D structural mesh and the original second 3D structural mesh further comprising determining one or more volume shapes resulting from differences between the adjusted second 3D structural mesh and the original second 3D structural mesh.

[0017] In some example embodiments, the method further comprising: identifying one or more implant regions of the adjusted second 3D structural mesh forming a boundary with the original 3D structural mesh, wherein the adjusted second 3D structural mesh within the boundary extends outwardly from the original 3D structural mesh; calculating one or more implant volume shapes for each of the identified one or more implant regions based on the volume generated in each implant region between the original second 3D structural mesh and the adjusted second 3D structural mesh therein; and outputting data representative of one or more implant volume shapes and identified implant regions for controlling manufacture of the corresponding one or more implants in relation to the bodily portion of the subject.

[0018] In some example embodiments, the method further comprising: identifying one or more removal regions of the adjusted second 3D structural mesh forming a boundary with the original 3D structural mesh, wherein the adjusted second 3D structural mesh within the boundary extends to within the interior of the original 3D structural mesh; calculating one or more removal volume shapes for each of the identified one or more removal regions based on the volume generated in each removal region between the original second 3D structural mesh and the adjusted second 3D structural mesh; outputting data representative of one or more removal volume shapes and identified removal regions for controlling removal of said removal regions from the bodily part of the subject.

[0019] In some example embodiments, the bodily portion of the subject comprises at least one from the group of: a head portion of the subject comprising at least a face or facial region of the subject; a neck portion of the subject comprising at least a neck region of the subject; a torso portion of the subject comprising at least a shoulder, chest, abdominal and / or pelvis region of the subject; a limb portion of the subject comprising at least an upper extremity or arm region including one or more of an upper arm, forearm or hand region of the subject, a lower extremity region including one or more of a hip, thigh, knee, leg, ankle or foot region of the subject.

[0020] In some example embodiments, the first 3D structure comprises data representative of one or more soft tissue regions associated with the bodily portion of the subject.

[0021] In some example embodiments, the second 3D structure comprises data representative of one or more bone or skeletal regions associated with the bodily portion of the subject.

[0022] In some example embodiments, the second 3D structure comprises data representative of one or more second soft tissue and / or facia regions associated with the bodily portion of the subject.

[0023] In some example embodiments, the first 3D structure comprises data representative of one or more soft tissue regions associated with the bodily portion of the subject; and the second 3D structure comprises data representative of either: a) one or more bone or skeletal regions associated with the bodily portion of the subject; b) one or more second soft tissue and / or facia regions associated with the bodily portion of the subject; or c) a combination of both a) and b).

[0024] In some example embodiments, the 3D medical model of the bodily portion of the subject is based on data representative of a 3D medical image captured from a medical imaging apparatus capable of capturing the first and second 3D structures of the bodily portion of the subject.

[0025] In some example embodiments, the medical imaging apparatus comprises one or more from the group of: ultrasound or ultrasonic scanning apparatus; Computerized Tomography (CT) scanning apparatus; Photon Counting CT scanning apparatus; Magnetic Resonance Imaging (MRI) apparatus; any other medical imaging apparatus capable of capturing the first and second 3D structures of the bodily portion of the subject.

[0026] In some example embodiments, the 3D photogrammetry data of the bodily portion of the subject is based on imaging data captured from one or more image capturing apparatus for generating 3D photogrammetry data representative of the external appearance of the bodily portion of the subject.

[0027] In some example embodiments, obtaining the 3D model further comprising: receiving, from an image capturing apparatus, first 3D photogrammetry data of the bodily portion of the subject requiring the one or more implants, the first 3D photogrammetry data defining an external 3D appearance of the bodily portion of the subject; receiving, from a medical image capturing apparatus, data representative of medical imagery of the bodily portion of the subject for generating the 3D medical model of the portion of the subject, the 3D medical model comprising the first 3D structure and the second 3D structure underlying the first 3D structure in relation to the bodily portion of the subject; and superimposing the first 3D photogrammetry data onto the 3D medical model.

[0028] In some example embodiments, the first 3D structure comprising a first 3D structural mesh of the outer surface of the first 3D structure, the first 3D photogrammetry data of the bodily portion comprising a first 3D photogrammetry mesh, wherein superimposing the 3D photogrammetry data onto the first 3D structural mesh further comprising: receiving a reference 3D mesh comprising a plurality of landmarked features associated with the bodily portion of the subject; identifying the plurality of landmarked features on each of the first 3D structural mesh and the first 3D photogrammetry mesh; generating a first reference 3D mesh based on superimposing the reference 3D mesh onto the first 3D structural mesh using the reference and identified landmarks of the reference 3D mesh and first 3D structural mesh; generating a second reference 3D mesh based on superimposing the reference 3D mesh onto the first 3D photogrammetry mesh using the reference and identified landmarks of the reference 3D mesh and first 3D photogrammetry mesh; and adjusting the second reference 3D mesh to match the first reference 3D mesh based on performing one or more mesh transformation operations; superimposing the first 3D photogrammetry mesh onto the first 3D structural mesh based on the one or more performed mesh transformation operations; and superimposing the 3D photogrammetry data onto the superimposed first 3D photogrammetry mesh to form the 3D model of the bodily portion of the subject.

[0029] In some example embodiments, obtaining the 3D model further comprising: receiving data representative of desired adjustments to the superimposed first 3D photogrammetry data in relation to the first and second 3D structures of the bodily portion of the subject; controlling adjustments to the superimposed first 3D photogrammetry mesh relative to the first 3D structure of the bodily portion of the subject until the desired outcome is reached taking into account the structural properties of at least the first 3D structure; and outputting the 3D model comprising at least the 3D photogrammetry data defining the desired external appearance of the bodily portion of the subject and the 3D medical model.

[0030] In some example embodiments, the method further comprising controlling the adjustments of the superimposed first 3D photogrammetry data in relation to the properties of the first and second 3D structures of the bodily portion of the subject by: identifying one or more implantable regions based on the positioning of the first 3D structure over the second 3D structure of the bodily portion of the subject and structural properties thereof; receiving data representative of desired adjustments in relation to one or more identified implantable regions; determining the feasibility of each desired adjustment in relation to the desired adjusted one or more implantable regions based on modelling the desired adjustments of the implantable regions in relation to the structural properties of the first and second 3D structures; in response to determining a desired adjustment of an implantable region is infeasible, indicating the feasibility of the desired adjustment and / or limiting the adjustment to a feasible adjustment for the implantable region, otherwise allowing the adjustment until the desired outcome is reached.

[0031] In some example embodiments, the first 3D structure comprising a first 3D structural mesh, the first 3D photogrammetry data of the bodily portion comprising a first 3D photogrammetry mesh, wherein the 3D photogrammetry mesh has been superimposed onto the first 3D structural mesh, and obtaining the 3D model further comprising: controlling adjustments to the superimposed first 3D photogrammetry mesh and corresponding first 3D photogrammetry data taking into account the structural properties of at least the first 3D structure of the bodily portion of the subject until a desired outcome is reached; and generating the 3D model based on data representative of: the adjusted 3D photogrammetry mesh and 3D photogrammetry data defining the desired external appearance of the bodily portion of the subject; and the 3D medical model comprising data representative of the first and second 3 structures and corresponding first and second 3D structural meshes.

[0032] In some example embodiments, the first 3D structure comprising a first 3D structural mesh, the first 3D photogrammetry data of the bodily portion comprising a first 3D photogrammetry mesh, wherein the 3D photogrammetry mesh has been superimposed onto the first 3D structural mesh, and obtaining the 3D model further comprising: controlling adjustments to the superimposed first 3D photogrammetry mesh and corresponding first 3D photogrammetry data taking into account the structural properties of at least the first 3D structure of the bodily portion of the subject until a desired outcome is reached; and generating the 3D model based on data representative of: the adjusted 3D photogrammetry mesh and 3D photogrammetry data defining the desired external appearance of the bodily portion of the subject; and the 3D medical model comprising data representative of the first and second 3D structures and corresponding first and second 3D structural meshes.

[0033] In some example embodiments, the method further comprising controlling the manufacturing one or more medical implants based on the output data representative of the one or more implants.

[0034] In some example embodiments, manufacturing one or more medical implants further comprising: 3D printing one or more medical implants using the data representative of the one or more implant volumes; and / or controlling a manufacturing process at an implant manufacturer for producing one or more medical implants using the data representative of the one or more implant volumes.

[0035] According to a second aspect, there is provided a non-transitory tangible computer-readable medium comprising data or instruction code, which when executed on one or more processor(s), causes at least one of the one or more processor(s) to perform the steps of the computer-implemented method of the first aspect or any preceding computer-implemented method definition.

[0036] According to third aspect, there is provided a system for generating one or more implants for a subject for use in cosmetic or reconstructive surgery, the system comprising one or more processors, a memory and a communication interface, the memory comprising program instructions that, when executed by the one or more processors, cause the system to perform operations comprising: obtaining a three-dimensional, 3D, model of a bodily portion of the subject, the 3D model comprising: 3D photogrammetry data representative of the bodily portion of the subject and defining a desired external appearance of the bodily portion of the subject; and a 3D medical model representative of the bodily portion of the subject, the 3D medical model comprising a first 3D structure and a second 3D structure underlying the first 3D structure in relation to the bodily portion of the subject; determining one or more volumes generated between the 3D photogrammetry data of the desired bodily portion of the subject and the second 3D structure taking into account the structural properties of at least the first 3D structure; and outputting data representative of the determined one or more volumes for controlling the manufacture of one or more corresponding implants for the subject.

[0037] The program instructions of the third aspect may also perform operations, steps, functionality according to any of the first and / or second aspect or any preceding method definition of the first and / or second aspects.

[0038] According to a fourth aspect, there is provided an apparatus comprising a processor, a memory unit and a communication interface, wherein the processor is connected to the memory unit and the communication interface, wherein processor, memory and communication interface are configured to implement the computer-implemented method according to preceding method definition of the first and / or second aspects.

[0039] In a fifth aspect, this specification describes a computer-readable medium (such as a non-transitory computer-readable medium) comprising program instructions stored.thereon for performing (at least) any preceding method or apparatus definition as described with reference to any of the first, second, third, and / or fourth aspects. Brief Description of the Drawings

[0040] Embodiments of the invention will be described, by way of example, with reference to the following drawings, in which:

[0041] Figure la is a schematic diagram illustrating an example implant generation system according to some embodiments of the invention;

[0042] Figure lb is a flow diagram illustrating an example implant generation process for generating implants according to some embodiments of the invention;

[0043] Figure ic is a flow diagram illustrating an example implant analysis process for use in the implant generation system of figure la according to some embodiments of the invention;

[0044] Figure id is a schematic diagram illustrating an example 3D model generation process for use in implant generation system of figure la according to some embodiments of the invention;

[0045] Figure le is a flow diagram illustrating an example 3D model generation process according to some embodiments of the invention;

[0046] Figure if is a flow diagram illustrating an example 3D mesh superposition process for use in the 3D model generation process of figure id according to some embodiments of the invention;

[0047] Figure ig is a flow diagram illustrating an example 3D model adjustment process for use in the 3D model generation process of figure id according to some embodiments of the invention;

[0048] Figure 2 is a schematic diagram illustrating another example implant generation system according to some embodiments of the invention;

[0049] Figure 3a is a schematic diagram illustrating another example 3D model generation system according to some embodiments of the invention;

[0050] Figure 3b is a schematic diagram illustrating another example 3D model adjustment apparatus according to some embodiments of the invention;

[0051] Figure 3c is a schematic diagram illustrating an example implant analysis apparatus according to some embodiments of the invention;

[0052] Figures 4a to 4d are schematic diagrams illustrating an example implant analysis process according to some embodiments of the invention;

[0053] Figure 5 is a schematic diagram of a computing system according to some embodiments of the invention; and

[0054] Figure 6 is a schematic diagram of a computer-readable medium according to some embodiments of the invention.

[0055] Common reference numerals are used throughout the figures to indicate similar features. Detailed Description

[0056] Example embodiments relate to systems, methods and / or computer programs for combining a 3D model of one or more bodily portions of a subject (e.g., CT / MRI scans of a patient’s face) and 3D photogrammetry data (e.g. 3D photographs / models) of the one or more bodily portions of the subject into a 3D photogrammetric model of the one or more bodily portions of the subject describing the relationship between the aesthetic shape, the overlying structure (e.g. soft tissue shape) and underlying structure (e.g. bone form or other underlying soft tissues) of the bodily portion of the subject. Changes to aesthetic shape of one or more regions of the 3D photogrammetric model and modelling the underlying structures maybe used to drive an implant design process. This enables corresponding changes to the overlying structure and / or underlying structures of the 3D photogrammetric model of the bodily portion of the subject (e.g., face) to be modelled and verified by taking into account the properties of the overlying and / or underlying structures (e.g. mass-spring modelling, finite element modelling, machine learning (ML) models and / or other modelling / analytical techniques). This modelling models the changes these structures may undertake when, constrained by their properties, they are adapted to the changes in aesthetic shape. This may involve adapting the underlying structural surface to increase / decrease in one or more regions that expand, stretch and / or compress the overlying structure to naturally form into the changed aesthetic shape. The modelling may include modelling the properties of the proposed implant materials in relation to adapting the underlying structure. Data representative of volumes and / or shapes required to achieve the aesthetic changes may be generated based on the differences between the adapted underlying structure and the original underlying structure in relation to the one or more regions.

[0057] Data representative of one or more of these volumes and / or shapes may be used to define the volume / shape of one or more implants that are designed specifically for the subject. This may be output in a suitable implant volume / shape format for manufacture / creation of the implant for the subject. Once manufactured / created, the one or more implants maybe applied during surgery to the corresponding region and underlying structure of the subject to achieve the aesthetic changes.

[0058] For example, data representing the implant volumes / shapes may be output in a suitable format for creation and production of implants according to the implant volumes / shapes. The creation and production of the implants may include, without limitation, for example sending the data representative of the implant volumes to a manufacturer for producing implants according to the implant volumes and materials required, or the data representative of the implant volumes maybe transmitted to a 3D printer for printing the 3D shapes of the implant volumes in accordance to the required implants and implant materials and medical standard.

[0059] Alternatively and / or additionally, data representative of one or more of the volumes and / or shapes may define how much of the underlying structure in the corresponding region maybe removed from the subject (e.g. assisting in, without limitation, for example osteotomies, orthognathic surgery, mandibuloplasty or other similar types of cosmetic / reconstructive / surgical operations). This may be output in a suitable volume / shape reduction format for use in providing an indication of which regions and how much of the underlying structure (e.g. bone structure) said regions of the subject maybe removed to achieve the aesthetic or desired changes. For example, the data representative of the volume to be removed from the second underlying structure may be used to control a robotic reconstructive surgical apparatus including image sensors and a surgical implement, which may be controlled to sense the location of the volume of the second structure of the subject and remove, using the surgical implement, the required volume from the second underlying structure of the bodily portion of the subject. Alternatively, the data representative of the volume may be used by an image analysis apparatus configured to analyse an image or video stream of the underlying structure of the subject (e.g. during reconstructive surgery) and provide an indication of whether or not the determined volume and shape thereof has been sufficiently removed to realise the desired bodily portion of the subject.

[0060] The bodily portion of the subject may include, without limitation, for example at least one or more bodily portions of a subject from the group of: a head portion of the subject including, for example, at least a face or facial region of the subject; a neck portion of the subject; a torso portion of the subject including, for example, at least a shoulder, chest, abdominal and / or pelvis region of the subject, and / or any other bodily portion of the torso of the subject; a limb portion of the subject; at least an upper extremity or arm region such as, for example, one or more upper arms of the subject, forearms of the subject, or hand regions of the subject; a lower extremity region including, for example, one or more of a hip of the subject, one or more thighs of the subject, one or more knees of the subject, a lower leg or calve region of the subject, one or more ankle or foot regions of the subject; and / or any other bodily portion of the subject suitable for cosmetic and / or reconstructive surgery requiring implants being applied to and / or removal of one or more underlying structures of the subject and the like that may change the aesthetic outcome of the bodily portion and the like and / or as the application demands.

[0061] The advantages of the systems, methods and / or computer programs described herein include providing a subject and their surgeon with a 3D model / simulation of realistic desired outcomes for the bodily portion of the subject. Further advantages of the systems, methods and / or computer programs described herein include, from the 3D model, determining the volume and shape of any implants required, and from an output implant format, medically manufacturing / creating said implants, and / or indicating an amount of underlying structure for removal from the subject, needed to induce anatomical normality or the desired anatomical changes rather than relying on real-time surgical guess-work, estimation, experimentation, and / or repeated corrective follow-up visits. This systems, methods, and / or computer programs herein provide an efficient and cost effective mechanism for improving desired outcomes for subjects and the like in relation to reconstructive and / or cosmetic surgery and the like.

[0062] Figure la is a schematic diagram illustrating an example implant generation system 100. The implant generation system includes a 3D model component 102, an implant analysis component 104 and an implant generation component 106 connected together. The 3D model component 102 is configured to obtain a 3D model of a bodily portion of the subject (e.g., a face, chest or lower leg, etc.), and provide a 3D model of a desired bodily portion of the subject to the implant analysis component 104. The 3D model of the desired bodily portion also includes 3D model of the underlying structures (e.g. soft tissue and bone structures) of the bodily portion of the subject. The implant analysis component 104 processes the 3D model of the desired bodily portion of the subject to analyse the original underlying structures of the subject and adapt these to conform the original 3D model of the bodily portion of the subject to the 3D model of the desired bodily portion of the subject. Differences between the original underlying structures and the adapted structures may be used to form one or more volumes. From this adaptation one or more volumes are determined and / or generated between the adapted and original underlying structures and data representative of the one or more volumes is output for use in generating implants associated with the determined one or more volumes.

[0063] In operation, the 3D model component 102 may receive a 3D medical model of the bodily portion of the subject with underlying 3D structures of the bodily portion of the subject, and 3D photogrammetry data of the bodily portion of the subject defining an external appearance of the bodily portion of the subject. The 3D model component 102 may be configured to transform and superimpose the 3D photogrammetry onto the 3D medical model to form a 3D model of the bodily portion of the subject that is medically accurate whilst also realistically representing the external appearance of the bodily portion of the subject. The resulting 3D model may include data representative of the 3D medical model and transformed 3D photogrammetry that has been superimposed thereon. The 3D model component 102 maybe further configured to control adjustment of one or more regions of the 3D photogrammetry of the 3D model of the bodily portion of the subject to a desired shape or size (e.g., adjust the cheek shape of the face, each breast of the chest or calve of a lower leg etc.). Once a desired shape or size of the bodily portion has been achieved, a 3D model representing the desired bodily portion may be output to the implant analysis component 104. The 3D model of the desired bodily portion of the subject may include data representative of at least the adjusted 3D photogrammetry representing the desired bodily portion of the subject and the 3D medical model of the bodily portion of the subject.

[0064] The implant analysis component 104, on receiving the 3D model of the desired bodily portion of the subject, and perform an analysis (e.g. modelling or simulation such as, without limitation, for example finite element analysis / modelling, machine learning (ML) models, and / or spring-mass analysis / modelling or other modelling, simulation and / or numerical analysis techniques) that processes and adapts the underlying 3D structures of the 3D medical model of the desired bodily portion of the subject to conform with the 3D photogrammetry of the 3D model of the desired bodily portion of the subject. From the results of this adaptation, the implant analysis component 104 is further configured to determine data representative of one or more volumes between the adapted underlying 3D structures of the 3D model that are required to realistically conform the 3D medical model of the bodily portion of the subject to the 3D photogrammetry data of the 3D model of the desired bodily portion of the subject. Data representative of the one or more volumes may be output for controlling an implant generation process for generating one or more implants conforming to the determined one or more volumes.

[0065] The implant generation process 106, receives data representative of one or more volumes for use in generating or creating one or more implants for the bodily portion of the subject. The data representative of the one or more volumes may be used to control the manufacture of one or more corresponding implants by an implant manufacture process. For example, the data representative of the one or more volumes maybe used to direct a 3D printer to print, using medical printing materials, and output one or more medical implants for use in reconstructive and / or cosmetic surgery on the bodily portion of the subject. The reconstructive or cosmetic surgery performed on the subject using said implants may result in the subject achieving the external appearance desired bodily portion resulting in fewer or no corrective surgeries and the like.

[0066] Figure lb is a flow diagram illustrating an example implant generation process 110 for generating one or more implants for use in reconstructive or cosmetic surgery in relation to the subject. The implant generation process 110 may include one or more of the following steps of:

[0067] In step 112, obtaining a 3D model of a bodily portion of the subject. The 3D model may include data representative of: 3D photogrammetry data representative of the bodily portion of the subject and defining a desired external appearance of the bodily portion of the subject, and a 3D medical model representative of the bodily portion of the subject. The 3D medical model includes data representative of a first 3D structure and a second 3D structure underlying the first 3D structure in relation to the bodily portion of the subject. For example, the first 3D structure may be soft tissue structures such as, without limitation, for example skin and muscle, whilst the second 3D structure underlying the first 3D structure may be bone, cartilage, or skull underlying or supporting the first 3D structure of skin and muscle above. Alternatively, the first 3D structure maybe a first soft tissue structure, e.g. skin, subcutaneous fat, muscle, and / or cartilage structures, and the second 3D structure may be a second soft tissue structure underlying the first soft tissue structure, e.g. further muscles, cartilage, or bone structures.

[0068] In step 114, determining one or more volumes generated between the 3D photogrammetry data of the desired bodily portion of the subject and the second 3D structure taking into account the structural properties of at least the first 3D structure.

[0069] For example, the determination may include, processing the 3D model of the desired bodily portion to determine the implant volumes required to achieve the desired bodily portion of the subject. This processing may include adjusting the original first and second 3D structures taking into account the structural properties of at least the first 3D structure and / or second 3D structure until the outer surface of the first 3D structure substantially coincides with the outer surface of the desired bodily portion, e.g. the desired 3D photogrammetry data. As an example, the first 3D structure may represent the soft facial tissue and the second 3D structure may representing the skull or jaw bone underlying the soft facial tissue. In another example, this processing may include an iterative algorithm configured to iteratively adjust the original first and second 3D structures taking into account the structural properties of at least the first 3D structure until the outer surface of the first 3D structure substantially coincides with the outer surface of the desired bodily portion. For example, the iterative algorithm maybe configured such that adjustments to the second 3D structure (e.g. bone of the bodily portion such as, without limitation, for example skull, cheek or jaw bone) may be made whilst changes to the first 3D structure (e.g. soft tissue of the bodily portion such as, without limitation, for example facial soft tissue) are modelled based on the structural properties of the first 3D structure and the adjusted second 3D structure due to these adjustments. This process of adjusting the second 3D structure and modelling the effect on the first 3D structure (e.g. compression, stretching, and elasticity of the soft tissues) may be iterated or simulated until the outer surface of first 3D structure substantially coincides with the outer surface of the 3D model of the desired bodily portion of the subject. Based on these adjustments, the final adjusted second 3D structure maybe compared with the original second 3D structure, where one or more volume shapes may have been created to achieve the required transformation of the first 3D structure. For example, one or more volumes between the adjusted second 3D structure and the original second 3D structure maybe calculated based on determining one or more volume shapes resulting from differences between the adjusted second 3D structure and the original second 3D structure.

[0070] For example, the first and second 3D structures may be represented by a first and second 3D structural mesh, and the desired 3D photogrammetry data may be represented by a 3D photogrammetry mesh that represents the 3D model of the desired bodily portion of the subject. Each of the first 3D structural mesh, second 3D structural mesh, and 3D photogrammetry mesh may each be a different polygon mesh representing the outer surface of the first 3D structure, the second 3D structure and 3D photogrammetry model, respectively. A polygon mesh may be described as a collection of vertices, edges and “faces” that define the outer surface or shape of a polyhedral object and may be used to represent an outer surface of a 3D object (e.g. outer surface of a first 3D structure, outer surface of a second 3D structure, and / or outer surface of a 3D photogrammetric model and the like). The “faces” usually consist of triangles (e.g., triangle mesh), quadrilaterals (e.g., quads), or other simple convex polygons (e.g., n-gons) since this simplifies rendering, or a mixture of different polygonal “faces” depending on the complexity of the 3D object / model, but may also be more generally composed of concave polygons, or even polygons with holes, and the like and / or as the application demands.

[0071] Thus, the first and second 3D structural meshes may be iteratively adjusted whilst modelling or simulating the effects of each of the adjustments on the first 3D structure taking into account the properties of the first 3D structure (e.g. soft tissue elasticity, compressibility, stretching and the like) until the first 3D structural mesh is sufficiently aligned or coincides with the desired 3D photogrammetry mesh of the desired bodily portion of the subject. Based on these adjustments, one or more volumes may be created that achieve the required transformation of the first 3D structural mesh to substantially coincide with the 3D photogrammetry mesh of the desired bodily portion. One or more volumes between the adjusted second 3D structural mesh and the original second 3D structural mesh may be calculated based on determining one or more volume shapes resulting from differences between the final adjusted second 3D structural mesh and the original second 3D structural mesh.

[0072] In step 116, outputting data representative of the determined one or more volumes for controlling the manufacture of one or more corresponding implants for the subject. The determined one or more volumes may be used by a manufacturing process to create one or more medical implants corresponding to said one or more volumes. These may then be applied during reconstructive and / or cosmetic surgery to the corresponding locations on the bodily portion of the subject to achieve the desired bodily portion of the subject. For example, the data representative of the one or more volumes maybe output and made into one or more implants for the subject by, without limitation, for example one or more manufacturing techniques, 3D printing techniques, and / or any other implant generation or creation process configured for creating the one or more implants for the subject. For example, the data representative of the one or more volumes may be output to a 3D printer for printing, using medical grade materials and under medical conditions, the corresponding one or more implants for the subject. In another example, the data representative of the one or more volumes may be transmitted to a suitable manufacturer of implants, and used to manufacture the one or more implants for the subject. Although several techniques are described for manufacturing the one or more implants, this is by way of example only and the invention is not so limited, it is to be appreciated by the skilled person that the data representative of the one or more volumes maybe used for creating or sculpting using any suitable technique for creating said one or more implants for the subject and / or as the application demands. These implants may then be applied during reconstructive or cosmetic procedures performed on the subject to achieve the desired bodily portion.

[0073] Figure lc is a flow diagram illustrating an example implant analysis process 120 for use in the implant generation system 100 of figure la. The implant analysis process 120 may be used to analyse the differences between the desired bodily portion and the original bodily portion of the subject for determining one or more volumes for use in generating one or more corresponding implants according to the one or more volumes. In this example, the 3D model of the desired bodily portion of the subject may include at last data representative of: 3D photogrammetry mesh representative of the desired bodily portion of the subject and defining the 3D shape of the desired external appearance of the bodily portion of the subject, and the 3D medical model representative of the bodily portion of the subject. In this case, the 3D medical model further includes at least data representative of a first 3D structure and a second 3D structure underlying the first 3D structure in relation to the bodily portion of the subject, and a first 3D structural mesh representing the outer surface of the first 3D structure and a second 3D structural mesh representing the outer surface of the second 3D structure. The implant analysis process 120 may include the following steps of:

[0074] In step 121, receiving the 3D model including at least a 3D photogrammetry mesh of the desired bodily portion of the subject, a first 3D structural mesh of the bodily portion of the subject, and a second 3D structural mesh of the bodily portion of the subject underlying first 3D structural mesh.

[0075] In step 122, performing one or more adjustments to the second structural mesh to deform the first 3D structure and the first 3D structural mesh towards the 3D photogrammetry mesh of the desired bodily portion of the subject based on the structural properties of the first 3D structure. For example, this may include modelling the structural properties of the first 3D structure as the first and second 3D structural meshes are adjusted or modified.

[0076] For example, this modelling may be performed using machine learning (ML) algorithms and the like trained for adjusting first and second 3D structural meshes taking into account properties of at least the first 3D structural mesh. For example, an ML model maybe based on one or more ML models from the group of: a neural network (NN) comprising a plurality of neural network layers, each neural network layer associated with a set of parameters / weights; a convolutional NN (CNN) comprising a plurality of convolutional layers; a transformer -based ML model comprising a first and second transformer encoder / decoder NNs; an encoder / decoder NN; and / or any other suitable ML model capable of being trained to determine whether desired adjustments made are feasible given a first 3D structural mesh, a second 3D structural mesh and a 3D photogrammetry mesh of the desired bodily portion. Training of the ML model may be based on obtaining a training dataset comprising a plurality of training data instances. Each training data instance comprising data representative of a first 3D structural mesh, second 3D structural mesh, desired 3D photogrammetry mesh, and appropriately feasible deformed first and second 3D structural meshes. For each training iteration of a plurality of training iterations the following maybe performed: one or more training data instances (or a batch of training instances) are applied to the ML model, in which the first, second and desired 3D photogrammetry meshes are input, and which outputs whether the deformations are feasible and / or outputs the deformed first and / or second 3D structural meshes; an estimation of a loss is performed based on a difference between the output deformed first and / or second 3D structural meshes and the corresponding first and second deformed 3D structural meshes of each of the one or more training data instances. The set of weights of the ML model may be updated based on the estimated loss. In each subsequent iteration of the plurality of training iterations further one or more training instances (e.g. further batches of training instances) are retrieved for applying to the ML model, estimating the loss and updating the weights of the ML model and the like. Training the ML model may stop once a stopping criterion is reached, e.g. an error threshold is met, or a maximum number of training iterations is reached, or other performance metric associated with the particular type of ML model is met. The trained ML model may then be used to determine one or more volumes for the implants based on the deformed second 3D structural mesh and the original second 3D structural mesh. Alternatively, this modelling may be performed using other iterative or numerical analytical tools such as, without limitation, for example finite element analysis, mass-spring models, or other types of numerical analytical tools configured for modelling and / or iteratively adjusting the first and second 3D structural meshes taking into account properties of at least the first 3D structural mesh, the first 3D structure associated with the first 3D structural mesh, and / or the second 3D structure / structural mesh until the first 3D structural mesh is substantially aligned / coincident with the 3D photogrammetric mesh of the desired bodily portion of the subject.

[0077] In step 123, it is determined whether the deformed / adjusted first 3D structural mesh matches or is sufficiently aligned / coincident with the 3D photogrammetry mesh of the desired bodily portion of the subject. If this is the case (e.g. Y), then the process 120 proceeds to step 124, otherwise (e.g. ‘N’) the process 120 proceeds back to step 122 for further iterative adjustments and the like. This iterative loop may continue until a stopping criterion is achieved, without limitation, for example a maximum number of iterations and / or until a certain error threshold is achieved between how close the first 3D structural mesh matches the 3D photogrammetry mesh of the desired bodily portion of the subject. However, if the first 3D structural mesh cannot converge towards the 3D photogrammetry mesh of the desired bodily portion of the subject within the tolerances associated with a match (or being sufficiently coincident), then this may indicate the desired bodily portion may not be achievable for the subject, but may be used to display the best matching or closest bodily portion to the desired bodily portion of the subject. Once confirmed, the process 120 may proceed to step 124.

[0078] In step 124, determining one or more volume shapes based on the differences between the adjusted second 3D structural mesh and the original second 3D structural mesh.

[0079] In step 125, outputting data representative of the one of more volume shapes for controlling manufacture of one or more corresponding implants.

[0080] Figure id is a schematic diagram illustrating an example 3D model generation system 130 for use with implant generation system 100 of figure la. The 3D model component 102 of figure la may further include a 3D image capture component 131a, a 3D medical model capture component 131b, a 3D image / medical model superposition component 132, a 3D model adjustment component 133, and a 3D model output component 134, which may be coupled to the implant generation process system 100 and implant analysis and generation components 104 and 106.

[0081] The 3D image capture component 131a may be configured to generate the 3D photogrammetry data defining the external appearance of the bodily portion of the subject. For example, the 3D image capture component 131a may include using one or more imaging capturing devices that capture one or more 2D photographs of the external appearance or texture of the bodily portion of the subject, which maybe processed into a 3D photogrammetry model of the bodily portion of the subject. In another example, a number of cameras spaced apart and positioned around the subject (e.g., 2-10 cameras) maybe used to capture multiple 2D pictures of the subject, which are processed and simultaneously and stitched together into a 3D picture or model of the subject to form the 3D photogrammetry data. Alternatively, a handheld camera may be moved around the bodily portion of the subject (e.g. subject’s head), which captures multiple 2D pictures or captures a 3D picture that maybe stitched together to form the 3D photogrammetry data. Although 2-10 cameras is described, this is by way of example only an the invention is not so limited, it would be appreciated by the person skilled in the art that any number of one or more cameras or a suitable number of cameras may be used and / or applied to image the subject from different angles / orientations and generate a 3D photogrammetry data of the bodily portion of the subject as long as the generated 3D photogrammetry data is good enough for use in generating a 3D model of the subject with accurate 3D photogrammetry data as the application demands.

[0082] In any event, the 3D image capture component 131a, may capture and generate 3D photogrammetry data that represents the 3D external texture / appearance of the bodily portion of the subject. This may be referred to herein as a 3D photogrammetry model. The 3D photogrammetry data includes data representative of a 3D photogrammetry model of the bodily portion of the subject, where the 3D photogrammetry data is transformed and combined with the 3D medical model of the bodily portion of the subject in 3D image / model component 132. Thus, a 3D model of the bodily portion of the subject incorporating both a 3D medical model and the 3D photogrammetry data / model of the bodily portion of the subject can be generated.

[0083] As well as generating the 3D photogrammetry data of the bodily portion of the subject by 3D image capture component 131a, 3D medical model capture component 131b may be configured to generate the 3D medical model of the bodily portion of the subject defining at least two structures of the bodily portion of the subject such as, for example the soft-tissue structure and underlying bone structure of the bodily portion of the subject. For example, the 3D medical model capture component 131b maybe configured to generate the 3D medical model from medical imaging apparatus (e.g. CT, MRI or ultrasonic scan or other suitable type of 3D medical scanning / imaging technology) that may image the bodily portion of the subject, where the first and second 3D structures are a 3D representation of soft tissues and / or bone associated with the bodily portion of the subject. For example, the bodily portion of the subject may be the head, and a CT scan of the subjects head may be performed to yield CT scanning data representative of the soft facial tissue and skull (e.g., cheek and jaw bones) of the subject. The CT scan data maybe processed to generate a 3D medical model of the head of the subject that includes a first 3D structure representing by the soft facial tissue and a second 3D structure representing the skull underlying the soft facial tissue.

[0084] The 3D image / model component 132, the 3D photogrammetry data may be transformed and superimposed over the outer surface of the 3D medical model (e.g. over the first 3D structure) to provide a realistic 3D representation of the bodily portion of the subject. The 3D image / model component 132 outputs a 3D model of the bodily portion of the subject that represents a medically accurate and realistic appearance of the bodily portion of the subject. The output 3D model includes at least data representative of the 3D photogrammetry data transformed and superimposed onto the 3D medical model.

[0085] The 3D model adjust component 133 may be configured to, once the 3D model of the bodily portion of the subject has been generated, receive adjustments from an operator or user may input adjustments to the 3D model to achieve a desired appearance for the subject. This may include receiving the adjustments and adjusting the 3D photogrammetry data that is superimposed over the 3D medical model by pulling, stretching, pushing various regions or data points of the bodily portion of the subject represented by the 3D photogrammetry data, which changes the appearance of the bodily portion of the subject (e.g. for a face, this may include changing the shape of the cheek bone area or jaw line / chin areas and the like). Thus the original 3D model maybe transformed into a 3D model of a desired bodily portion of the subject, which includes the 3D photogrammetry data transformed with the desired adjustments, and also the original 3D medical model. This may then be processed to determine suitable implant volumes that will achieve, from reconstructive or cosmetic, the desired bodily portion for the subject.

[0086] The 3D model output component 134 maybe configured to output data representative of the 3D model of the desired bodily portion of the subject. The 3D model of the desired bodily portion of the subject including at least: data representative of: 3D photogrammetry data representative of the bodily portion of the subject and defining a desired external appearance of the bodily portion of the subject, and a 3D medical model representative of the bodily portion of the subject. The 3D medical model includes data representative of a first 3D structure and a second 3D structure underlying the first 3D structure in relation to the bodily portion of the subject. As an example, the 3D photogrammetry data may further include a 3D photogrammetry mesh and texture map of the bodily portion of the subject, the first 3D structure may further include a first 3D structural mesh representing the outer surface of the first 3D structure, and the second 3D structure may further include a second 3D structural mesh representing the outer surface of the second 3D structure, which interfaces with the first 3D structure. The 3D model may be output to the implant analysis process component 104 of figure la.

[0087] Figure te is a flow diagram illustrating an example 3D model generation process 140 for use in example 3D model generation system 130 of figure if. The example 3D model generation process 140 includes the following steps of:

[0088] In step 141, obtaining data representative of first 3D photogrammetry data defining an external appearance and / or texture of a bodily portion of the subject.

[0089] For example, in some embodiments, the first 3D photogrammetry data may include a 3D photogrammetry model of the bodily portion of the subject. The 3D photogrammetry model further includes a 3D photogrammetry mesh representing the outer surface of the bodily portion of the subject and a corresponding texture map of the bodily portion of the subject, which when combined forms a 3D photogrammetry model of the bodily portion of the subject.

[0090] In step 142, obtaining a second 3D model of the bodily portion of the subject, the second 3D model defining a first 3D structure and a second 3D structure underlying the first 3D structure in relation to the bodily portion of the subject. The first 3D structure may further include a first 3D structural mesh representing the outer surface of the first 3D structure, and the second 3D structure may further include a second 3D structural mesh representing the outer surface of the second 3D structure, which interfaces with or borders the first 3D structure.

[0091] For example, the second 3D model may be derived from medical imaging data of the bodily portion of the subject captured by a medical imaging apparatus and may form a 3D medical model. For example, the medical imaging apparatus may include, without limitation, for example, a CT scanner, MRI scanner, ultrasonic scanner and / or any other medical imaging apparatus suitable for outputting data for generating a 3D medical model of the bodily portion of the subject. The medical imaging data may be further processed to generate a first 3D structure and second 3D structure underlying the first 3D structure. For example, the first 3D structure may include soft-tissue structures of the bodily portion (e.g. muscles, fat, blood vessels, veins, skin of the bodily portion of the subject). The second 3D structure underlying the first 3D structure may include, without limitation, for example bone structures and / or other soft-tissue structures associated with the bodily portion of the subject. As an option or in some embodiments, the 3D medical model may be further processed and include with the first 3D structure, data representative of a first 3D structural mesh corresponding to the outer surface of the first 3D structure, and also include with the second 3D structure, data representative of a second 3D structural mesh corresponding to the outer surface of the second 3D structure.

[0092] In step 143, superimposing the 3D photogrammetry data onto the second 3D model.

[0093] In step 144, adjusting the appearance of the superimposed first 3D photogrammetry data in relation to the bodily portion of the subject until a desired outcome is reached.

[0094] In step 145, outputting a 3D model of a desired bodily portion of the subject, where the 3D model includes data representative of at least: 3D photogrammetry data of the desired bodily portion of the subject, and a 3D medical model representative of the original bodily portion of the subject including data representative of at least: the first 3D structure and the second 3D structure underlying the first 3D structure of the bodily portion of the subject. In some embodiments, the 3D photogrammetry data includes a 3D photogrammetry mesh and corresponding texture map, and the 3D medical model further includes a first 3D structural mesh corresponding to the outer surface of the first 3D structure and a second 3D structural mesh corresponding to the outer surface of the second 3D structure.

[0095] Figure if is a flow diagram illustrating an example 3D superposition process for use in the 3D model generation process of figure id according to some embodiments of the invention. In this example, the 3D photogrammetry data includes a 3D photogrammetry mesh and a corresponding texture map. As well, the 3D medical model includes a first 3D structure (e.g., soft tissue) and a second 3D structure (e.g., bone) underlying or supporting the first 3D structure. The first 3D structure further includes a corresponding first 3D structural mesh representing the outer surface of the first 3D structure. The second 3D structure further includes a corresponding second 3D structural mesh representing the outer surface of the second 3D structure. The 3D superposition process 150 transforms the 3D photogrammetry data onto the 3D medical model such that the 3D photogrammetry data represents a realistic and medically accurate outward appearance of the bodily portion of the subject. The 3D superposition process 150 includes the following steps of:

[0096] In step 151, retrieve a landmarked 3D reference mesh, the 3D photogrammetry mesh of the 3D model of the bodily portion of the subject, and the first 3D structural mesh of the 3D model of the bodily portion of the subject.

[0097] The landmarked 3D reference mesh may be a standard or uniform 3D mesh representing the outer surface of a standard shape of the bodily portion (e.g., a standard head shape, standard chest shape, standard leg shape etc.). The 3D reference mesh may be of a lower resolution compared with the first 3D structural mesh and / or the 3D photogrammetry mesh. This provides the advantage of reducing the computational load whilst maintaining suitable accuracy when superimposing the 3D photogrammetry mesh onto the first 3D structural mesh. Alternatively, the 3D reference mesh may be of a similar or higher resolution compared with the first 3D structural mesh or 3D photogrammetry mesh to enable hyper accurate superpositioning. In any event, the 3D reference mesh, when landmarked, is used to assist in transforming and mapping the 3D photogrammetry mesh onto the first 3D structural mesh during the superpositioning to provide an accurate as possible 3D representation of the external appearance of the 3D model of the bodily portion of the subject.

[0098] The 3D reference mesh includes a set of designated landmark mesh points representing important features or landmarks associated with the shape of the bodily portion. For example, landmark mesh points associated with a face may include, without limitation, for example points around the eyes, points around the forehead or brow / nose bridge, points around the tip and / or nostrils of the nose, points around the chin, jawline and / or cheek bones and the like. These may be used to lock the reference mesh to corresponding landmark mesh points on the 3D photogrammetry and first 3D structural meshes.

[0099] In step 152, landmarking the 3D photogrammetry mesh and first 3D structural mesh by identifying mesh points corresponding to the landmarked 3D reference mesh points on the 3D photogrammetry mesh and first 3D model structural mesh. For example, each of the 3D photogrammetry mesh and first 3D model structural mesh maybe analysed and processed (e.g., using 3D graphics feature identification software and / or ML algorithms trained to identify landmark points, and the like) to identify the mesh points or regions that correspond to the landmark mesh points of the landmarked 3D reference mesh. The identified mesh points for each of the 3D photogrammetry and first 3D structural meshes maybe annotated, labelled and / or mapped to the corresponding landmark points of the 3D reference mesh.

[00100] In step 153, generating a first 3D reference mesh by performing a first set of mesh transformation operations on the original landmarked reference mesh to deform the original landmarked 3D reference mesh to substantially match the surface of the landmarked 3D photogrammetry mesh, e.g., corresponding landmark points match. For example, for each of the landmarked points of the landmarked 3D reference mesh, a first set of one or more transformation operations may be determined that transform these landmarked points to match the landmarked points of the landmarked 3D photogrammetry mesh. This first set of transformation operations may also be used to assist in transforming the other points of the landmarked reference mesh until the surface of the landmarked 3D reference mesh substantially coincides or matches the surface of the landmarked 3D photogrammetry mesh. This transformation process generates a first 3D reference mesh that substantially matches the 3D surface of the 3D photogrammetry mesh.

[00101] In step 154, generating a second 3D reference mesh by performing a second set of mesh transformation operations on the original landmarked 3D reference mesh to deform original landmarked 3D reference mesh to substantially match the surface of the landmarked first 3D structural mesh so that corresponding landmark points match. For example, for each of the landmarked points of the landmarked 3D reference mesh, one or more second set of transformation operations may be determined that transform these landmarked points to match the landmarked points of the landmarked first 3D structural mesh. This second set of transformation operations may also be used to assist in transforming the other points of the landmarked 3D reference mesh until the surface of the landmarked 3D reference mesh substantially coincides or matches the surface of the landmarked first 3D structural mesh. This transformation process generates a second 3D reference mesh that substantially matches the 3D surface of the first 3D structural mesh.

[00102] In step 155, adjusting the first reference 3D mesh to match the second reference 3D mesh based on determining and performing a third set of one or more mesh transformation operations. For example, a third set of mesh transformation operations may be determined that operate on the first reference 3D mesh to transform each of the mesh points of the first reference 3D mesh to match the corresponding mesh points of the second reference 3D mesh. This is possible because the first and second reference 3D meshes have the same number of mesh points as the original landmarked reference mesh. As well, both the first and second 3D reference meshes have the same landmark points as the original landmarked reference mesh. The transformation operations that are used to transform the mesh points of the first reference 3D mesh to coincide with the mesh points of the second reference 3D mesh maybe stored as a third set of mesh transformation operations. The third set of mesh transformation operations may be used to merge or superimpose the 3D photogrammetry mesh onto the first 3D structural mesh.

[00103] In step 156, superimposing the first 3D photogrammetry mesh and data (e.g. corresponding texture mapping) onto the first 3D structural mesh based on the third set of one or more mesh transformation operations. Given that the first 3D reference mesh was aligned with the 3D photogrammetry mesh, the third set of one or more mesh transformation operations may be applied to the mesh points of the 3D photogrammetry mesh for transforming these mesh points towards the mesh points of the first 3D structural mesh. The texture mapping may also be transformed along with the superimposed 3D photogrammetry mesh using the third set of mesh transformation operations and the like.

[00104] Figure ig is a flow diagram illustrating an example 3D model adjustment process 160 for use in the 3D model generation process 140 of figure te or the 3D model adjust component 133 of figure id. The 3D model adjustment process 160 may include the following steps of:

[00105] In step 161, receive 3D model with 3d photogrammetry mesh superimposed onto first 3D structural mesh of 3D medical model of bodily portion of the subject. For example, the 3D model maybe received from step 156 or 143 and the like of processes 150 and 140, respectively. [ooio6] In step 162, receiving data representative of adjustments to one or more regions of the 3D photogrammetry mesh of the bodily portion of the subject. For example, the external appearance (e.g. 3D photogrammetry mesh and texture map) of the 3D model may be displayed to a user or operator on a device. The user or operator may manipulate one or more regions of the displayed 3D model that change the shape and external appearance of the 3D model as desired via one or more predefined manipulation operations (e.g. pull or push etc.), which are received as adjustments to be made to the corresponding regions of the 3D photogrammetry mesh. These adjustments may be used to adjust or transform the 3D photogrammetry mesh of the received 3D model to form a 3D photogrammetry mesh of the desired bodily portion of the subject.

[00107] In step 163, confirming whether the received adjustments to said one or more regions are feasible. For example, prior to displaying the adjustments, the received adjustments to the 3D photogrammetry mesh may be assessed as to whether they are feasible to be applied to one or more of the regions of the 3D model based on the corresponding regions of the first and / or second 3D structures and their properties such as, without limitation, for example elasticity of the skin or outer surface of the first 3D structure, compression of the first 3D structure and / or whether there is sufficient portions of the second 3D structure underlying the first 3D structure that may support an implant to realise the desired bodily portion of the subject. This may be a rule based assessment based on standard skin elasticity properties, closeness of adjustments to landmarked features (e.g. eye sockets, nose, and / or mouth etc.), the type of adjustment and the like. Alternatively, the assessment may include modelling and adjusting the first and second 3D structures to determine whether sufficient volumes may be generated to form implant volumes between the first and second 3D structures of the 3D model (e.g. finite element analysis may be performed, or spring-mass analysis, and / or ML algorithms trained to assess whether changes to the first 3D structural mesh may be made to meet the desired 3D photogrammetry mesh.

[00108] In step 164, displaying update of 3D model representing the desired bodily portion of the subject after feasible adjustments have been applied to the 3D photogrammetry mesh. For example, only those feasible adjustments may be applied and displayed, with infeasible adjustments not being made or an indication being displayed as to why such adjustments are not feasible. This maybe overridden though depending on the experience of the user and / or operator when making said adjustments. In which case, all adjustments maybe applied to the 3D photogrammetry mesh and the resulting 3D model of the desired bodily portion of the subject may be displayed.

[00109] In step 165, it is determined whether further modifications or adjustments to the 3D model are received. For example, the user maybe asked whether further adjustments are to be made. If this is the case (e.g. Y), then the process 160 proceeds to step 162 for receiving the further adjustments and controlling whether they are feasible or not and the like, otherwise (e.g. ‘N’), where the user may indicate finalisation of the adjustments made to the 3D photogrammetry mesh for output and / or storage, the process 160 proceeds to step 166 for displaying, storing and / or outputting a finalised 3D model of the desired bodily portion of the subject based on the adjustments 3D photogrammetry mesh that were iteratively performed in steps 162-165.

[00110] In step 166, outputting data representative of a finalised 3D model of a desired bodily portion of the subject, where the finalised 3D model includes at least data representative of: the 3D photogrammetry data of the desired bodily portion of the subject as displayed, and the 3D medical model representative of the original bodily portion of the subject. The 3D photogrammetry data of the desired bodily portion of the subject further includes the final adjusted 3D photogrammetry mesh corresponding to the desired bodily portion of the subject as displayed, and the corresponding texture mapping of the desired bodily portion of the subject. The 3D medical model of the original bodily portion of the subject including: the first 3D structure and corresponding first 3D structural mesh, the second 3D structure underlying the first 3D structure and the corresponding second 3D structural mesh of the bodily portion of the subject. The finalised 3D model of the desired bodily portion of the subject may be stored for further adjustments and / or modifications, or for later processing by implant generation process 100 or implant analysis process 102 and the like. The finalised 3D model of the desired bodily portion of the subject may be displayed to the subject for approval prior to further processing and / or prior to further adjustments using process 160. For example, the finalised 3D model of the desired bodily portion of the subject (e.g. once approved by the subject and / or user), maybe input to the implant generation process 100 or implant analysis process 102 for determining data representative of one or more volumes for use in controlling the creation, production and / or manufacture of corresponding implants and the like. [oom] Figure 2 is a schematic diagram illustrating another example implant generation system 200. The implant generation system 200 is based on the implant generation system too and processes 110,120,130,140,150,160 as described with reference to figures la to lg, the components and steps of which may be further modified / combined as described with reference to figure 2. The implant generation system 200 includes one or more processors or processing devices 220a-220g, one or more memory units or storage devices 222, one or more communication interfaces, user input / output systems (e.g. displays) 224a-224b, in which the one or more processors or processing devices 22oa-22og are coupled or connected to the one or more memory units or storage devices 222, communication interfaces and user input / output systems 224a-224b. Although the implant generation system 200 is described herein with reference to the bodily portion of the subject (e.g., a patient) being a head with a focus on the face of the subject, this is by way of example only and the invention is not so limited, it is to be appreciated by the skilled person that the implant generation system 200 may be applied to any other bodily portion of the subject as the application demands. In this example, data from CT scans (e.g. 3D medical model) and 3D photographs (e.g. photogrammetry data) of the face of the subject are combined into a realistic 3D model of the subject that describes the relationship between overlying soft tissue shape (e.g. first 3D structure) and underlying skull form (second 3D structure). Implant design and volume is determined by modelling the changes to the soft tissue shape of the face using, without limitation, for example finite element modelling, mass-spring models, numerical analysis techniques, or even an ML algorithm trained to determine volumes based on the properties and / or limitations of the overlying and underlying structures.

[00112] An advantage of the image generation process 200 is provided in the manner of merging the CT scan and 3D photogrammetry data of the face of the subject but retaining the necessary data to represent the subject in a realistic and lifelike manner that allows accurate modification of the external appearance of the face and, where necessary, generate the geometry for the necessary facial implant to achieve the features being represented based on the modifications. Further advantage of the image generation system 200 is that the desired changes to the face can be modelled or simulated for feasibility and for determining one or more volumes and / or shapes for implants to be created, where the system 200 outputs data representative of the determined one or more volumes / shapes that may be used to produce / manufacture one or more corresponding implants that achieve the changes needed to induce anatomical normality.

[00113] The implant generation system 200 includes a patient / capture processing system 224 including a photogrammetry processing apparatus 201 including processing device 220a. In this case, the photogrammetry processing apparatus 201 may be configured to capture a plurality of 2D images of the face 202 of the subject from different angles. The plurality of 2D images of the face 202 of the subject may be stored in a data storage system 220 (e.g. patient data storage). The data storage system 220 may be a database management system or other type of data storage repository and the like. The plurality of 2D images of the face 202 may be processed by a 3D photogrammetry pipeline process 203 on processing device 220b for converting the plurality of 2D images of the head / face 202 to create a 3D head exterior, which forms a 3D exterior model with 3D photogrammetry mesh and texture mapping 204, also referred to as 3D photogrammetry model 204 of the face of the subject. The 3D photogrammetry model 204 includes data representative of the 3D photogrammetry mesh describing the 3D shape of the head / face and texture mapping photogrammetry data that may be superimposed, or shrink wrapped onto the 3D photogrammetry mesh to provide a realistic 3D model of the face of the subject.

[00114] In some embodiments, the 3D photogrammetry model 204 may be communicated and displayed by a face manipulation process 205 on one or more other processing devices 22of (e.g. a tablet or other computing device) for display and used by a user or operator to manipulate and change the shape of the face to a desired facial representation of the subject. The 3D model is useful for previewing or experimenting with the possible changes that might be made to the face of the subject. For example, the 3D model may be manipulated by rotating the view of the head from different angles, in which the face manipulation process 205 may include face deformation tools (e.g. interactive controls / tools) enabling changes to be made to the face (e.g. pulling / pushing regions of the 3D photogrammetry mesh) and realistically adjusting the texture mapping to display the desired 3D model of the face of the subject. This may also enable side-by-side comparison between the original 3D model and an adjusted 3D model that may represent a desired face of the subject.

[00115] The patient / capture processing system 224 further includes a medical imaging processing apparatus 206 including processing device and / or hardware 220c. In this case, the medical imaging processing apparatus 206 may be configured to capture and / or process a CT / MRI image scan data 207 of the face of the subject for creating a 3D hill head anatomical model of the face of the subject, also referred to herein as a 3D medical model. The CT / MRI scan data 207 may be stored in data storage system 220 for further processing. The CT / MRI scan data 207 of the face of the subject may be processed by a 3D medical model pipeline process 208 on processing device 22od for converting the CT / MRI scan data 207 to create a 3D head anatomical model, which forms a 3D medical model 209 of the face / head of the subject. The 3D medical model pipeline process 208 maybe configured to process the CT / MRI scan image data to determine at least a first 3D structure (e.g. soft-tissues / skin of the face) and a second 3D structure (e.g. bone or skull of the head / face of the subject) underlying the first 3D structure. The first 3D structure may include data representative of a first 3D structural mesh representing the outer surface of the first 3D structure. The first 3D structure may also include data representative of the properties of the first 3D structure, e.g. elasticity, compressibility of the skin, fat and / or muscles of the soft-tissue. The second 3D structure may include data representative of a second 3D structural mesh representing the outer surface of the second 3D structure. The second 3D structure may also include data representative of the properties of the second 3D structure, e.g. hardness of the bone and / or cartilage and the like.

[00116] A implant processing system 226 including processing device 220g and a display is configured, in this example, to perform or control facial modelling, facial manipulation and facial implant processes as described herein. For example, the bodily portion is the face, and so a facial modelling process may be configured to retrieve the 3D Photogrammetry model 204 and the 3D medical model 209 (e.g. CT Scan / MRI of head (3D)) from storage 222 and merge the 3D Photogrammetry model 204 with the 3D medical model 209 (e.g. CT Scan / MRI of head (3D)) into an accurate medical representation of the face of the subject including accurate texture mapping of the exterior appearance of the subject over the first structure of the 3D medical model 209, as well as, accurate anatomical structures of the head / face of the subject (e.g. first and second structures) and displayed as a 3D model 210 of the face of the subject. The displayed 3D model 210 maybe stored in storage 222 as 3D model data 211.

[00117] The implant processing system 210 may also be further configured for manipulating the exterior appearance of the facial features of the 3D model 210 using a face manipulation process that enables a user or operator to input one or more manipulations to change the shape of the face of the 3D model 210 to a desired facial representation of the subject. The face manipulation process maybe coupled to a modelling engine 215 (e.g. rule-based engine, spring-mass analysis engine, or finite element analysis engine, or ML algorithm) that models, in real-time or non-real-time, and controls the input manipulations to display the realistic changes to the 3D model 210 depending on the properties of the first and second structures for display. The modelling engine 215 may also be configured to determine whether any changes input by the user are feasible in relation to the manipulated regions of the surface of the 3D model 210. Displaying the manipulations using the 3D model 210 is useful for previewing or experimenting with the possible realistic changes that might be made to the face of the subject. The face manipulation process may include 3D manipulation tools that are used to input manipulations (or adjustments) that are received from a user by the system 200 to control the alteration of the surface of the face of the 3D model 210. For example, the 3D photogrammetry mesh of the 3D model 210 may be adjusted by the input manipulations. The 3D model may be further manipulated by rotating the view of the head from different angles, in which the face manipulation process may include face deformation tools (e.g. interactive controls / tools) enabling changes (or adjustments) to be made to the face (e.g. pulling / pushing regions of the 3D photogrammetry mesh) and realistically adjusting the texture mapping to display the desired changes to the 3D model 210 of the face of the subject. This may also enable side-by-side comparison between the original 3D model 210 and the desired adjustments made to the 3D model 210 of the face of the subject. The face manipulation process allows, for example, the patient (or subject), user and / or surgeon to adjust the face using the realistic 3D model 210. The face manipulation process software is configured to accurately display the adjusted face using the 3D model 210 for visualisation by the user and / or subject. Although the face manipulation process may use modelling engine 215 to perform modelling in real-time, this is byway of example only and the invention is not so limited, it is to be appreciated by the skilled person that the modelling engine 215 may perform the modelling in non-real-time or off-line whilst the facial manipulations are performed in real-time and / or approximated in real-time to enable the subject, user and / or surgeon to visualize the possible manipulations and / or changes to the 3D model 210 to achieve the desired adjustments and desired face for the subject. The non-real-time modelling to determine whether the desired manipulations / adjustments are feasible, and to determine implant shape may be performed off-line or in non-real-time, after a set of adjustments for manipulating the angles of the face / head in real time have been performed using facial manipulation tools and the like. Alternatively or additionally, modeling engine 215 may perform a lower complexity modelling (e.g. low level massspring analysis or ML technique, or facial manipulation rule-based algorithm) capable of being performed in real-time of the desired adjustments / manipulations to the 3D model 210 to determine whether the desired adjustments / manipulations are feasible for the particular subject given the properties of the first 3D structure and underlying second 3D structure of the 3D model of the subject. Once the desired 3D model has been approved, a higher-complexity modelling process (e.g. finite element analysis, or higher level mass-spring analysis or other ML technique) may be performed in non-real-time when using the desired 3D model to more accurately determine the one or more volume shapes for creating one or more implants for the subject.

[00118] Once the 3D model 210 of the face of the subject has been manipulated and desired adjustments to the 3D model 210 have been made, the desired adjustment to the 3D model 210 maybe stored as a desired 3D model 212 for further processing. The desired 3D model 212 may include data representative of a manipulated / adjusted / desired 3D photogrammetry mesh or a 3D photogrammetry mesh of the desired face of the subject. This has been changed from the original 3D photogrammetry mesh of the original 3D model 210 based on the manipulations made by the user or operator. The desired 3D model 212 also includes data representative of the original 3D medical model and the first 3D structure and second 3D structure underlying the first 3D structure, along with first and second 3D structural meshes, respectively and the like.

[00119] Once the desired 3D model 212 of the face of the subject has been generated and / or displayed, the desired 3D model 212 maybe used to generate implants for the subject. Implant generation system 228 includes an implant creation pipeline 213 that is configured to generate the implant shape using a modelling engine 215 (e.g. finite element analysis, mass-spring analysis, other suitable numerical analysis techniques, and / or ML algorithms) to calculate the necessary changes in the features of the skull (e.g. second 3D structure) so the outer surface of skin of the soft-tissue (e.g. the first 3D structural mesh, which is the outer surface of the first 3D structure) matches the chosen external features of the 3D photogrammetry mesh of the desired 3D model 212. As an example, the implant creation pipeline 213 may be configured to make use of a modelling engine 215 (e.g. finite element analysis, mass-spring analysis or other suitable analysis or ML technique) to determine one or more volumes or geometries within the desired 3D model 212 that may be used for generating or creating implants, which once created maybe applied to the face of the subject during reconstructive surgery so the subject’s face maybe anatomically altered achieve the appearance of the desired 3D model 212. For example, finite element analysis (or mass-spring analysis process) may be applied to model and calculates the changes in the underlying bone (e.g. second 3D structure) required to change the original soft-tissue surface (e.g. first 3D structural mesh) produce the manipulated soft tissue surface associated with the desired 3D model 212 (e.g. the manipulated / adjusted / desired 3D photogrammetry mesh). Both skull anatomy and soft tissue elasticity place constraints on the size, position and shape of possible implants. The modelling engine 215 may use these factors to as constraints in the finite element analysis process (or mass-spring analysis process) to ensure that the implant design is clinically feasible and to also determine the geometries of the implants. As an example, the implant creation pipeline 213 may use the modelling engine 215 such as, for example, a finite element analysis engine (or mass-spring analysis engine), to calculate the geometry of the implant to match the external features of the desired 3D model 212. The modelling engine 215 may iteratively adjust the surface of the second 3D structure of the 3D medical model (e.g. the second 3D structural mesh) and model any changes to the first 3D structure of the 3D medical model that would result due to the adjustments to the second 3D structure and using the structural properties of the first 3D structure. The second 3D structure maybe iteratively adjusted based on increasing outwardly (or inwardly) one or more surface regions of the surface of the second 3D structure. These adjustments to the second 3D structure maybe to model the required changes to the first 3D structure resulting from the adjustments to the second 3D structure, which further change the outer surface of the first 3D structure (e.g. the first 3D structural mesh). The adjustments are iterated until the first 3D structural mesh has been changed to match or substantially match the desired 3D photogrammetry mesh (e.g. within an error threshold).

[00120] For example, the second 3D structural mesh of the second 3D structure maybe adjusted and further deformed and / or increased outwardly in one or more regions during the iterative adjustments, where each adjustment to the second 3D structural mesh is translated to the first 3D structural mesh based on modelling how the first 3D structure and first 3D structural mesh deforms in relation to adjustments made to the second 3D structural mesh and the structural properties of the first 3D structure. The second 3D structure and second 3D structural mesh may be iteratively adjusted until the outer surface of the first 3D structure, i.e. the deformed / changed first 3D structural mesh, substantially aligns or matches with the exterior surface of the desired 3D model 212, i.e. the desired 3D photogrammetry mesh. That is, the second 3D structure may be iteratively adjusted until the first 3D structural mesh of the first 3D structure substantially aligns or matches with the desired 3D photogrammetry mesh representing the desired shape of the face of the subject from the desired 3D model 212.

[00121] Once alignment is reached or no further adjustments are possible (e.g. a stopping criterion such as a minimum error threshold is reached, or a maximum number of iterations is reached, and the like), the implant creation pipeline 213 is configured to determine the differences between the final adjusted second 3D structure and the original second 3D structure. For example, determine the differences between the final adjusted second 3D structural mesh and the original second 3D structural mesh, which result in one or more volume shapes. These differences may result in one or more volume shapes or geometries being determined between the outer surface of the final adjusted second 3D structure (i.e. final adjusted second 3D structural mesh) and the outer surface of the original second 3D structure (i.e. original second 3D structural mesh). The data representative of the one or more volumes or geometries may be output and subsequently used to control generation of one or more corresponding implants. For example, data representative of the resulting one or more implant geometries can be output in suitable file format for input into computer aided design / computer aided manufacture software of an implant manufacturer for manufacturing implants of the required geometry. For example, data representative of each of the volumes maybe sent in a suitable format to a manufacturer for use in controlling the manufacturing process for producing implants having substantially the same shape and volume. Alternatively, the data representative of each determined volume may be transmitted to a 3D printer for printing one or more implants having substantially the same shape and / or volume. The 3D printer being a medical grade 3D printer operating under medical grade conditions with medical grade implant printing materials and the like.

[00122] In addition, should any adjustments be made to the volume or geometry of an implant by the manufacturer and / or required by surgical processes (e.g. screw fittings or mounting plates / fixings) prior to the manufacture of the implant, the adjusted volume / geometry of the implant maybe re-input to the face manipulation process along with the 3D model 210 or 211 for a final visualisation and / or adjustment prior to approving each implant for manufacture or production (e.g. 3D printing, machining, implant molding processes, implant manufacturing processes, and the like). Further adjustments can then be made to the outer surface of the soft tissue or first 3D structural mesh and the above processes repeated to achieve the best possible implant shape. Thus, the best possible implant shape that allows a subject to achieve, via reconstructive surgery, the desired 3D model 212 of the face may be performed through repeated cycles of the above procedures iteratively. This provides the advantage of allowing the manufacture of a custom made, patient specific, implant that is designed to provide a more predictable surgical outcome via a more efficient and predictable procedure.

[00123] Figure 3a is a schematic diagram illustrating another example 3D model generation system 300 that may be used in the implant generation systems too and 200 and processes thereof. In this example, the bodily portion of the subject (or patient) is the head / face of the subject. Although the 3D model generation system 300 is described with respect to the head / face of the subject, this is by way of example only and the invention is not so limited, it is to be appreciated by the skilled person that the 3D model generation system 300 is applicable to any bodily portion of the subject including, without limitation, for example, the neck, torso, chest, abdominals, thigh / buttocks, lower leg, calves, ankles and / or feet of the subject, and / or any other suitable bodily portion of the subject that may be operated on during cosmetic and / or reconstructive surgery. In this example,

[00124] a 3D model of the head / face of a subject is generated that has a medically accurate or realistic external appearance of the face of the subject, but which also includes the underlying 3D anatomical features such as soft-tissue structures and / or bone / cartilage structures, which are also medically accurate.

[00125] In order to get the 3D model of the head of the subject with a realistic external appearance that is aligned with anatomical features of the head of the subject, the 3D model generation system 300 is configured to perform various processing operations on a CT scan 301a and 3D Photogrammetry data 302a. In operation 301, a CT scan 301a of the head / face of the subject may be received. The CT scan 301a may be generated by a CT scanner and saved in a Digital Imaging and Communications in Medicine (DICOM) format, which is the standard for the communication and management of medical imaging information and related data. The DICOM CT scan imagery 301a may be processed using one or more image processing algorithms to generate skin and bone structures 301b and 301c and corresponding skin and bone non-homologous meshes 3Oid and 3Oie, respectively, that, when combined form a 3D medical model. For example, a first 3D model representing the soft-tissue structure of the face may be generated, where the first 3D model 301 includes a first 3D structure 301b (e.g. soft-tissues) and a first 3D structural mesh 3Oid representing the outer surface (e.g. skin) of the first 3D structure 301b. As well, a second 3D model representing the underlying bone structure 301c (e.g. skull) of the face may be generated from the CT scan data 301a, where the second 3D model includes a second 3D structure 301c (e.g. skull / bone structures) and a second 3D structural mesh 3Oie representing the outer surface (e.g. outer surface of the skull / bone) of the second 3D structure 301c. The first and second 3D structural meshes 3Oid and 3Oie may be non-homologous meshes or arbitrary meshes that describe the surface of the first and second 3D structures 301b and 301c as efficiently as possible.

[00126] In addition, in operation 302, a 3D photogrammetric model may be generated from 3D photogrammetry data 302a and / or received that captures the external appearance of the head / face of the subject. The 3D photogrammetry data may be processed to generate a 3D photogrammetry mesh 302b representing the outer surface of the face of the subject and a texture mapping 302c representing the outer appearance / texture of the head of the subject, when combined they may form a 3D photo model of the head of the subject. For example, the texture mapping 302b may be shrink-wrapped over the 3D photogrammetry mesh 302b when generating the 3D photo model of the head of the subject. The 3D photo data 302a may be captured by a imaging capturing apparatus, which may include a number of cameras spaced apart and positioned around the subject (e.g. typically 5 cameras) that capture a 2D or 3D pictures simultaneously, and in which image processing is performed to determine a 3D photo data by transformations, stitching / knitting the captured images of the subject together. Alternatively, a handheld camera may be used that is moved around the subject’s head and captures 2D or 3D pictures. Any other image capturing apparatus may be used that is capable of generating a 3D photogrammetry data of the head of the subject (or any other bodily portion) that includes a 3D photogrammetry mesh 302b describing the outer surface of the subject’s head and a texture mapping 302c of the subject’s head representing the skin / texture of the subject. However, the 3D photogrammetry mesh 302b that may be generated is also a non-homologous mesh that may be different to the first structural mesh.

[00127] Each 3D surface of the 3D model may be represented by meshes, which are essentially represented in 3D space by Cartesian (x, y, z) coordinates or points and different shaped polygons or lines between the points joined together to describe the shape of said each 3D surface in 3D space. However, the meshes generated from the 3D photo or CT scans means that the meshes are non-homologous as they are usually generated in a manner that is most efficient to create the 3D shape. Each point in the mesh does not correspond to a particular anatomical point, nor does each point in the mesh correspond to any other point in another mesh. For example, the mesh points of the 3D photogrammetry mesh 302b are generally or mostly different to the mesh points of the first 3D structural mesh 3O2d. This can result in unacceptable inaccuracies when attempting to apply or superimpose the texture mapping 302c to the 3D medical model inaccurate. In order to generate a medically accurate 3D model 309a that is also realistic in appearance to the head of the subject, the 3D photogrammetry mesh 302b is required to be accurately superimposed onto the first 3D structural mesh 3Oid of the 3D medical model. Thus, in operation 303, a landmarked reference mesh 303a is created or retrieved in which a set of mesh points on the landmarked reference mesh each correspond to different anatomical landmark or point on a standard 3D model of the head (or other bodily part). For example, the set of landmark mesh points may correspond to mesh points associated with anatomical landmarks such as salient points, lateral and medial campus of the head, mouth, tip of nose, eye sockets, eyebrow, bridge, forehead, check bones, nostrils, any other part or feature of the face that is common between faces and may be used as a landmark. In addition, image processing maybe performed to identify the anatomical landmarks represented by the set of landmark mesh points on the 3D photogrammetry mesh 302b and the first 3D structural mesh 3Oid (or CT scan mesh).

[00128] In operation 304a, a first deformation mapping is defined for deforming or transforming the reference mesh 303a onto the first 3D structural mesh 3Oid using the set of landmark mesh points of the reference mesh 303a and the identified set of landmark points of the first 3D structural mesh 3Oid. This results in a first deformed reference mesh 306a, which is a homologous landmarked reference mesh that is coincident or substantially coincident with the first 3D structural mesh 3Oid. This may then be used to define a 3D structural-reference mesh mapping between the mesh points of the first 3D structural mesh 3Oid and the mesh points of the first deformed reference mesh 306a. In operation 304b, a second deformation mapping is defined for deforming or transforming the reference mesh 303a onto the 3D photogrammetry mesh 302b using the set of landmark mesh points of the reference mesh 303 and the identified set of landmark points of the 3D photogrammetry mesh 302b. This results in a second deformed reference mesh 306b, which is a homologous landmarked reference mesh that is coincident or substantially coincident with the 3D photogrammetry mesh 302b. This may then be used to define a 3D photogrammetry-reference mesh mapping between the mesh points of the 3D photogrammetry mesh 302b and the mesh points of the second deformed reference mesh 306b.

[00129] In operation 306, the first 3D structural mesh 3Oid (e.g. CT scan skin mesh) is used as the ground truth as this is the most accurate mesh and so the 3D photogrammetry mesh 302b is deformed or transformed to fit onto the first 3D structural mesh 3Oid using the first and second reference meshes 306a and 306b and the corresponding deformation mappings. This is possible because there is a known correspondence between the mesh points of the first deformed reference mesh 306a and the second deformed reference mesh 306b. Thus, a spatial mapping from the 3D photogrammetry mesh 302b to the first 3D structural mesh 3Oid maybe determined by: a) generating set of deformation mappings or operations based on the first and second sets of deformations to transform the mesh points of the second reference mesh 306b onto the corresponding mesh points of the first reference mesh 306a; and b) using this set of deformation mappings along with the 3D photogrammetry-reference mesh mapping to generate a 3D photogrammetry spatial mapping that transforms the mesh points of the 3D photogrammetry mesh 302b to fit the first 3D structural mesh 301b of the 3D medical model; and c) transforming the 3D photogrammetry mesh 302b based on the 3D photogrammetry spatial mapping, to form a second 3D photogrammetry mesh 3O2d. Once the second 3D photogrammetry mesh 3O2d has been transformed and is aligned with the first 3D structural mesh 3Oid, in operation 307, the texture mapping 302c is transformed applied or overlaid onto the second 3D photogrammetry mesh 3O2d using the applicable mappings to form transformed texture mapping 3O2e. In operation 308, a textured homologous landmarked skin mesh coincident with the first 3D structural mesh (e.g. skin mesh of the CT scan) is generated from the transformed 3D photogrammetry mesh 302b and texture mapping 3O2e and combined with the 3D medical model to form 3D model 309a.

[00130] The 3D model 309a of the head of the subject may include at least data representative of: the 3D medical model including the first 3D structure 301a and corresponding first 3D structural mesh 301b, the second 3D structure 301c and the corresponding second 3D structural mesh 3Oie; and the transformed 3D photogrammetry data including the second 3D photogrammetry mesh 302b and the corresponding transformed texture mapping 302c.

[00131] Figure 3b is a schematic diagram illustrating another example 3D model adjustment system 310 that maybe used in the implant generation system too of figures la to ig or by face manipulation process of implant generation system 200 of figure 2 and the like. The 3D model adjustment system 310 includes a 3d model manipulation apparatus 311 that may receive the 3D model 309a of the head of the subject, and display it to a user. The 3D model 309a is a realistic representation of the head of the subject and anatomically accurate. The 3D model manipulation apparatus 311 receives adjustment inputs from the user and controls how the face of the 3D model 309a may be adjusted in a manner that is cosmetically feasible. This is achieved by controlling how the 3D photogrammetry mesh 302b is beformeb from user inputs by a suite of tools that allow the user or operator to abjust the 3D mobel 309a of the face to a besireb shape of the face. The suite of tools may inclube facial manipulation operations (e.g. bobily portion manipulation operations) that may be useb to pull, push, anb beform the shape of the surface of the face to a besireb shape.

[00132] As an option, the 3D model adjustment apparatus 311 may be configured to limit the adjustments to the 3D model 309a to only those regions of the face that may support implants and / or removal of one or more first or second 3D structures of the face. That is the 3D model adjustment apparatus 311 controls the manipulations to only those realistic modifications that may be cosmetically performed. For example, only certain selectable regions of the face may allowed to be manipulated such as nose, check bone area, jaw line, chin or any other area in which the soft-tissue is supported by bone. For example, a cheek bone region may be selectable and configured to allow manipulations within a bounded area around the cheek bone based on predetermined measurements from other facial features such as the nose or eye socket. The selectable regions of the face of the 3D model 309a may be based on one or more rules in relation to where an implant maybe placed and / or where parts of the second 3D structure may be removed and the like. For example, the boundary may not extended to within a predetermined threshold (e.g. tern) of the eye socket, and similarly for the nose and the like. Similar limitations may be applied to other regions such as chin, jaw line and the like. Thus, a user or operator may select an area to “operate on” for manipulating the facial features of the 3D model 309a. For example, a cheek region 312a may be selected for manipulation, where the manipulation is controlled by the 3D model adjustment apparatus 311 to be confined to the selected region 312a. In this example, the user, having selected a cheek region 312a, may input a pull action that pulls a region of the cheek region 312a outwards (e.g. see after adjustment). The 3D model adjustment apparatus receives the pull input in relation to the selected region 312a and, in response, is configured to adjust the corresponding region 312a of the 3D photogrammetry mesh 3O2d and the texture mapping 3O2e of the 3D model 309a in a corresponding fashion. After adjustment, the adjusted 3D model 309a is displayed to the user with the adjusted 3D photogrammetry mesh / texture. The user may then input further one or more adjustments, in which the 3D adjustment apparatus 311 performs the corresponding manipulations / adjustments to the 3D photogrammetry mesh / texture until a desired shape of the face is achieved with the desired 3D model 313 being displayed to the user. The resulting adjusted 3D photogrammetry mesh and texture mapping form the desired 3D photogrammetry mesh 3O2f and texture mapping 302g. It is noted that the manipulated regions of the face of the desired 3D photogrammetry mesh 3O2f are no longer aligned with the first 3D structural mesh 30id, this is because of the adjustments made to the facial features of the 3D model 309a.

[00133] Thus, the desired 3D model 313 of the head of the subject may include at least data representative of: the 3D medical model including the first 3D structure 301a and corresponding first 3D structural mesh 3Oid, the second 3D structure 301c and the corresponding second 3D structural mesh 3Oie; and the desired 3D photogrammetry data including the desired 3D photogrammetry mesh 3O2f and the corresponding desired texture mapping 302g. The desired 3D model 313 may be used by implant generation system 100 or 200 of figures la to lg and / or 2 for generating one or more implants that may be applied to the subject to achieve the desired face shape.

[00134] Figure 3c is a schematic diagram illustrating another example implant analysis 320 including implant analysis component 321 that receives the desired 3D model 313 and outputs data representative of one or more volumes or geometries that may be used to design / manufacture corresponding implants 3233-323^ The implant analysis component 321 receives the desired 3D model 313 of the head of the subject, which may include at least data representative of: the 3D medical model including the first 3D structure 301a and corresponding first 3D structural mesh 3Oid, the second 3D structure 301c and the corresponding second 3D structural mesh 3Oie; and the desired 3D photogrammetry data including the desired 3D photogrammetry mesh 3O2f and the corresponding desired texture mapping 302g.

[00135] In this example, the implant analysis component 321 uses finite element analysis (FEA) or any other suitable methodology (e.g., mass-spring model / analysis, volumetric analysis, ML modelling and the like) to determine the one or more volumes. In this example, an FEA methodology is described. For example, the structural properties of the skin and soft-tissues are well known (e.g. elastic viscosity properties skin and soft tissue), where skin can be deformed more than soft-tissue. Alternatively or additionally, the structural properties of the first and / or second 3D structures of the desired 3D model 313 may be estimated from the medical scan data of the subject that were taken (e.g. CT / MRI scans) and used to generate the first and second 3D structures and corresponding first and second 3D meshes. Thus, deformations of the first 3D structure maybe modelled based on deformations made to the second 3D structure. For example, deform the second 3D structure and mesh 30idc and 301c (e.g., bone) and model the effects that the deformation has on the first 3D structure and first 3D structural mesh 301 and 30id and determine whether the resulting deformed first 3D structural mesh (e.g. the deformed skin) aligns with the desired 3D photogrammetry mesh 3O2f. This deformation / modelling may be repeated iteratively by continuing to deform the second 3D structure and mesh and modelling the resulting deformation on the first 3D structure and mesh until the deformed first 3D structural mesh is aligned with the desired 3D photogrammetry mesh 3O2f (e.g. until a stopping criterion such as, without limitation, for example error between the deformed first 3D structural mesh and desired 3D photogrammetry mesh reaches an error threshold, or number of iterations is reached etc.), then the implant volumes or geometries 323a and 323b may be determined, where the volumetric differences between the original second structural mesh 3Oie (e.g. undeformed bone) and the deformed second structural mesh (e.g. deformed bone) form the one or more implant volumes, shapes or geometries 323a and 323b that are required.

[00136] For example, the desired 3D photogrammetry mesh 3Oie represents the desired deformation that the soft-tissues of the first 3D structure need to undergo in order for the first 3D structural mesh 3Oid to align with the desired 3D photogrammetry mesh 3O2f. The finite element analysis may iteratively adjust the shape of the second structural mesh 3Oie (e.g. the bone) while at the same time, in each iteration, modelling how the adjustments deform first 3D structure (e.g. the overlying soft tissues) and corresponding first 3D structural mesh 3Oid and whether the corresponding deformed first 3D structural mesh is in alignment or fits with the desired 3D photogrammetry mesh 3O2f. This iterative process is continued until alignment of the first 3D structure mesh with the desired 3D photogrammetry mesh 3O2f is determined to have been reached. This may be determined based on a stopping criterion such as a minimum error threshold calculated between the 3D photogrammetry mesh 3O2f and the deformed first 3D structural mesh and the like.

[00137] Once alignment is reached, the implant volumes or geometries required are determined by calculating the differences in the volumes between the deformed second 3D structural mesh and the original second 3D structural mesh 3Oie may be determined. The resulting volumes may be used to define the implant volumes / geometries required. Data representative of the resulting volumes may be output in a format for use in controlling an implant manufacturing process to design and make implants with the required implant volumes and / or geometries.

[00138] In another example, after the desired 3D photogrammetry mesh 3Oif has been generated by a user deforming (manipulating / adjusting) the 3D photogrammetry mesh 301b using face manipulation or sculpting tools, the implant volume(s) maybe determined based on the desired 3D photogrammetry mesh 3Oie, the first 3D structural mesh 3Oid and the second 3D structural mesh 3Oie. This maybe achieved using, without limitation, for example an finite element analysis (FEA) method to simulate and derive one or more implant volumes given the desired 3D photogrammetry mesh 3Oif and the originally generated first and second 3D meshes 3Oid and 3Oie, as well as a range of soft tissue and / or bone properties associated with the first and second 3D structures. FEA provides the advantage of accurately modelling different materials and structure, which improves the accuracy of determining accurate implant volumes from which implants may be created / made. The determination of implant volumes may be determined by FEA in non-real time, i.e. offline, after the desired 3D photogrammetry mesh has been approved and / or finalised.

[00139] The FEA method may be applied to the first and second 3D structural meshes (e.g. skin and bone meshes) to generate a prediction of required displacement of the second 3D structural mesh (e.g., skull / bone mesh) that causes the first 3D structural mesh to align with the desired 3D photogrammetry mesh taking in to account the structural properties (e.g., soft tissue and / or bone properties) of at least the first 3D structure in relation to the displacement of the second 3D structural mesh. The FEA method may iteratively be performed to displace the second 3D structural mesh whilst deforming the first 3D structural mesh taking into account the properties of at least the first 3D structure with respect to the displacement of the second 3D structural mesh. The FEA method can then be used to with the other meshes to derive an implant volume / mesh.

[00140] Input parameters and material properties for the FEA method may include, without limitation, for example Young’s modulus of the first and / or second 3D structures, Poisson ratio of the first and / or second 3D structures, viscoelasticity of the first and / or second 3D structures and / or any one or more other suitable input properties / values / boundary conditions associated with the first and / or second 3D structures. For example, when the second 3D structure is the skull / bone, the following ranges may be applied to the FEA method in relation to the second 3D structure: Young's Modulus for the bone: a Young’s Modulus value approximately in the range of 5000 MPa to 15000 MPa; Poisson ratio for bone: a Poisson ratio value approximately in the range of 0.2 to 0.4. When the first 3D structure is the soft-tissue, the following ranges may be applied to the FEA method in relation to the first 3D structure: Young's Modulus for the soft-tissue: a Young’s Modulus value approximately in the range of 0.1 MPa to 1 MPa; Poisson ratio for soft-tissue: a Poisson ratio value approximately in the range of 0.45 to 0.5; Viscoelasticity for soft-tissue: a viscoelasticity value approximately in the range of 30% to 95%.

[00141] Although various ranges for Young’s modulus, Poisson ratio, and viscoelasticity of the first and / or second 3D structures have been described herein, this is by way of example only and the invention is not so limited, it is to be appreciated by the skilled person that other values for the Young’s modulus, Poisson ratio, and viscoelasticity of the first and / or second 3D structures of the bodily portion of the subject may be measured and / or determined and used in the FEA method when modelling the first and second 3D structures / meshes, where such values for Young’s modulus, Poisson ratio, and may be within the above ranges and / or even outside the above ranges as measured / determined for that particular subject or the bodily portion of that particular subject and / or as the application demands. Although the input parameters and material properties for the FEA method include Young’s modulus, Poisson ratio and viscoelasticity, this is by way of example only and the invention is not so limited, it is to be appreciated by the skilled person that one or more other properties of the materials associated with the first and / or second 3D structures and / or implant materials, maybe input and / or applied in the FEA method and / or in addition to Young’s modulus, Poisson ratio, viscoelasticity associated with the first and / or second 3D structures and the like as the application demands.

[00142] The FEA method receives as input data representative of the desired 3D photogrammetry mesh, the first 3D structural mesh and second 3D structural mesh and structural properties of at least the first 3D structure. Taking the desired 3D photogrammetry mesh, and applying the FE method (e.g. in an iterative manner) to deform the second 3D structural mesh (e.g. bone mesh) and cause deformation of the first 3D structural mesh based on the structural properties of the first and second 3D structures, the resulting deformed second 3D structural mesh’s displaced position may be calculated along with a deformed first 3D structural mesh based on the structural properties of the first and second 3D structures. This may be iterated until the deformed first 3D structural mesh coincides or meets the desired 3D photogrammetry mesh within a certain error threshold or stopping criterion. During the FEA method, boundary conditions should be set in two directions, outwardly from the second 3D structure to the first 3D structure and longitudinally (e.g. for facial 3D model, these two directions may be from the back of the head facing forwards and from the top of the head towards the floor). Once the FEA method has reached a solution, the difference between the deformed second 3D structural mesh (e.g. deformed bone mesh) and the original second 3D structural mesh (e.g. bone mesh) is then used to determine the difference in volume between the deformed second 3D structural mesh (e.g. deformed bone mesh) and the original second 3D structural mesh (e.g. bone mesh). The resulting volume or volumes may be used as the volumetric shapes that form the one or more implants.

[00143] For more accuracy in the FEA method, as an option, the structural properties of the implant material may also be taken into account during the FEA method in relation to the displacement of the second 3D structure. This is because the volume associated with the displacement of the second 3D structure forms the implant volume. Additionally or alternatively as an option, should the second 3D structure be softer than bone or be a combination of bone and cartilage / muscle (e.g., the underlying second 3D structure may be ribs / muscle / cartilage of the chest, whilst first 3D structure comprises the pectoral muscle and / or subcutaneous fat and soft-tissues associated with a breast) or have similar properties to that of the first 3D structure (e.g., the first 3D structure is soft-tissue and the second 3D structure is also soft-tissue underlying the first 3D structure (i.e. a calf implant between calf muscles)), then the structural properties of the second 3D structure may be taken into account by the FEA method as both first and second 3D structures may be deformed or compressed when an implant is inserted therebetween.

[00144] Although FEA is described herein, this is by way of example only and the invention is not so limited, it is to be appreciated by the skilled person that FEA is one of many suitable analysis / modelling / processing techniques that maybe applied such as, without limitation, for example mass-spring analysis, one or more ML models trained to estimate implant volumes given a desired 3D model and the original 3D model, and / or any other type of process or technique for estimating implant volumes, modifications thereto, combinations thereof, and / or as the application demands.

[00145] Although in this example, the implant analysis component 321 was described as using FEA methodology, this is by way of example only and the invention is not so limited, it is to be appreciated by the skilled person in the art that any other suitable methodology maybe applied such as, without limitation, for example FEA, mass-spring model / analysis, volumetric analysis, ML modelling and / or any other type of numerical methodology or analysis methodology suitable to determine the one or more volumes, combinations thereof, modifications thereto and / or as the applications demand. The type of methodology may depend on the application requirements and / or computation speed, e.g. real-time computation may require a lower complexity modelling / analysis for determining feasibility of deformations / manipulations to the 3D photogrammetry mesh, whereas non-real time computation may allow higher complexity modelling / analysis such as FEA and / or other more accurate modelling for determining implant volumes and the like. That said, lower complexity modelling / analysis may also be accurate enough to enable determination of implant volumes and the like, as implants maybe reshaped prior to reconstructive operations and the like.

[00146] Alternatively or additionally, the implant analysis component 321 may be further modified to use a mass-spring model / analysis methodology instead of using the FEA methodology. For example, the mass-spring model (MSM) approach is based on constructing a framework of points connecting from the second 3D structural mesh (e.g. skull / bone mesh) to corresponding points on the first 3D structural mesh (e.g. the skin surface, or original 3D photogrammetry mesh) using a system of damped springs. Each point is attributed a mass, so that as deformations are made to the first 3D structural mesh (e.g. the skin surface) moving the connected points of the first 3D structural mesh, the corresponding connected points of the second 3D structural mesh also move accordingly, which deform the second 3D structural mesh. Once the first 3D structural mesh has been deformed to match or substantially match the desired 3D photogrammetry mesh, the resulting deformed second 3D structural mesh is used with the original second 3D structural mesh to determine the one or more volumes required for creating / making the implants. The implant analysis component 321 may be further modified to use a mass-spring model / analysis methodology instead of using the FEA methodology. For example, the mass-spring model (MSM) approach is based on constructing a framework of points connecting from the second 3D structural mesh (e.g. skull / bone mesh) to corresponding points on the first 3D structural mesh (e.g. the skin surface, or original 3D photogrammetry mesh) using a system of damped springs. Each point is attributed a mass, so that as deformations are made to the first 3D structural mesh (e.g. the skin surface) moving the connected points of the first 3D structural mesh, the corresponding connected points of the second 3D structural mesh also move accordingly, which deform the second 3D structural mesh. Once the first 3D structural mesh has been deformed to match or substantially match the desired 3D photogrammetry mesh, the resulting deformed second 3D structural mesh is used with the original second 3D structural mesh to determine the one or more volumes required for creating / making the implants.

[00147] The MSM method uses the second 3D structural mesh (e.g. bone surface mesh) and the first 3D structural mesh (e.g. skin surface mesh) to derive an evenly distributed system of points therebetween that will be used to attach springs between the second 3D structural mesh and first 3D structural mesh (e.g. bone and skin surfaces). In effect, the MSM methodology approximates / simulates the anisotropic behaviour of the first 3D structure (e.g. soft tissue) and / or deformation of the first 3D structural mesh using suitable dampening values for the springs whilst ensuring the volume between the first 3D structural mesh and second 3D structural mesh is maintained. As the first 3D structural mesh is deformed (e.g. towards the desired 3D photogrammetry mesh) the spring constraints on each of the points connected to the second 3D structural mesh will ensure that these points connected to the bone surface deform and move with the deformed first 3D structural mesh. Once the first 3D structural mesh coincides or meets the desired 3D photogrammetry mesh, the difference between the deformed second 3D structural mesh (e.g. deformed bone mesh) and the original second 3D structural mesh (e.g. bone mesh) is then used to determine the difference in volume between the deformed second 3D structural mesh (e.g. deformed bone mesh) and the original second 3D structural mesh (e.g. bone mesh). The resulting volume or volumes may be used as the volumetric shapes that form the one or more implants.

[00148] Alternatively, rather than using iterative or simulation / modelling techniques to calculate how to deform the second 3D structural mesh (e.g. bone / skull mesh) given a desired 3D photogrammetry mesh (e.g. deformed skin surface mesh) as described with reference to the implant analysis system 321 in figure 3c, which calculates deformation of the second 3D structural mesh after the desired 3D photogrammetry mesh has been performed, changes to the second 3D structural mesh may be made at the same time as changes to the 3D photogrammetry mesh until the desired 3D photogrammetry mesh is achieved. Although in figure 3c the first 3D structural mesh may be deformed to meet the desired 3D photogrammetry mesh, this is by way of example only and the invention is not so limited, it is to be appreciated by the skilled person that these and / or other techniques may be applied in which, when the 3D photogrammetry mesh (or first 3D structural mesh) is deformed in figure 3b by the user until a desired 3D photogrammetry mesh is achieved, corresponding deformations may be made to the second 3D structural mesh to generate a deformed second 3D structural mesh, which can be used to determine volumes for creating / making implants.

[00149] For example, the MSM methodology may be applied in the 3D model adjustment system 310 of figure 3b, which maybe used in the implant generation system too of figures la to ig or by face manipulation process of implant generation system 200 of figure 2 and the like. In the 3D model adjustment system 310, the 3D model manipulation apparatus 311 may receive the 3D model 309a of the head of the subject, and display it to a user. The 3D model 309a is a realistic representation of the head of the subject and anatomically accurate. The 3D model manipulation apparatus 311 receives adjustment inputs from the user and controls how the face of the 3D model 309a maybe adjusted in a manner that is cosmetically feasible. This is achieved by controlling how the 3D photogrammetry mesh 302b (or first 3D structural mesh 3Oid) is deformed from user inputs by a suite of tools that allow the user or operator to adjust the 3D model 309a of the face to a desired shape of the face. The suite of tools may include facial manipulation operations (e.g. bodily portion manipulation operations) that may be used to pull, push, and deform the shape of the surface of the face to a desired shape. The MSM methodology may be applied to connect points on the second 3D structural mesh 3Oie with corresponding points on the first 3D structural mesh 3Oid (or 3D photogrammetry mesh 3O2d), which are fitted the spring constraints and the like. Thus, whilst the user is adjusting the 3D model 309a and hence deforming the 3D photogrammetry mesh 3O2d and / or first 3D structural mesh 3Oid, the second 3D structural mesh 3Oie is being likewise deformed as described above with respect to the MSM methodology. Thus, once the user has finished with the adjustments and the 3D photogrammetry mesh 3O2d has been adjusted into a desired 3D photogrammetry mesh, likewise with the corresponding first 3D structural mesh, then the deformed second 3D structural mesh may be used to determine the volumes between the deformed second 3D structural mesh and the original second 3D structural mesh for use in creating / making the required implant volumes.

[00150] In another example, using iterative or simulation / modelling techniques to calculate howto deform the second 3D structural mesh (e.g. bone / skull mesh) given a desired 3D photogrammetry mesh (e.g. deformed skin surface mesh) and calculating deformation of the second 3D structural mesh after the desired 3D photogrammetry mesh has been performed, changes to the second 3D structural mesh maybe made at the same time as changes to the 3D photogrammetry mesh until the desired 3D photogrammetry mesh is achieved.

[00151] In another example, a volumetric approach may be performed during operation of the 3D model adjustment system 310 of figure 3b in which when a section on the 3D photogrammetry mesh 3O2d of the 3D model 309a of the bodily portion of the subject (e.g. a head model) is selected for editing, a ray may be cast from the selected section that locates the nearest portion of the second 3D structural mesh (e.g. bone mesh) and then the volume of that affected portion of the second 3D structural mesh (e.g. bone structure) is deformed.

[00152] For example, in the 3D model adjustment system 310, the 3D model manipulation apparatus 311 may receive the 3D model 309a of the head of the subject, and display it to a user. The 3D model 309a is a realistic representation of the head of the subject and anatomically accurate. The 3D model manipulation apparatus 311 receives adjustment inputs from the user and controls how the face of the 3D model 309a maybe adjusted in a manner that is cosmetically feasible. This is achieved by controlling how the 3D photogrammetry mesh 302b (or first 3D structural mesh 3Oid) is deformed from user inputs by a suite of tools that allow the user or operator to adjust the 3D model 309a of the face to a desired shape of the face. The suite of tools may include facial manipulation operations (e.g. bodily portion manipulation operations) that may be used to pull, push, and deform the shape of the surface of the face to a desired shape. The volumetric methodology as described above maybe applied whilst the user is adjusting the 3D model 309a and hence deforming the 3D photogrammetry mesh 3O2d and / or first 3D structural mesh 3Oid, the second 3D structural mesh 3Oie is being likewise deformed. For example, when a user attempts to make a deformation to the 3D photogrammetry mesh 3O2d using the suite of tools including bodily portion manipulation operations, a ray is cast from the position of the scene’s camera to the point on the 3D photogrammetry mesh 3O2d being deformed, and projecting from this point on the 3D photogrammetry mesh along a ray vector towards the second 3D structural mesh (e.g. bone mesh). If this ray vector intersects with the second 3D structural mesh, mesh points within range of where the ray intersects and if their normals are pointing towards the camera, are displaced by the same amount as the deformation to the 3D photogrammetry mesh 3O2d. This may be iterated as the user selects and deforms further portions of the 3D photogrammetry mesh 3O2d for deformation, where the deformed second 3D structural mesh is further deformed accordingly. The region of points around where the ray vector intersects on the deformed second 3D structural mesh may be adjusted to include a larger region of points or a smaller region of points that are adjusted based on the deformation. Alternatively or additionally, the region of points around where the ray vector intersects may have a gradient fall-off on the amount of distance or deformation that is applied to those points of the deformed second 3D structural mesh. The gradient falloff may be applied by instead of moving all the points in the region of points by the same distance deformation as on the 3D photogrammetry mesh 3O2d, the deformation distance is proportional to gradient fall off value. Thus, once the user has finished with the adjustments and the 3D photogrammetry mesh 3O2d has been adjusted into a desired 3D photogrammetry mesh, likewise with the deformed second 3D structural mesh, then the deformed second 3D structural mesh may be used to determine the volumes between the deformed second 3D structural mesh and the original second 3D structural mesh for use in creating / making the required implant volumes.

[00153] Figures 4a to 4d are schematic diagrams illustrating an example implant adjustment and analysis processes 400,410 420 and 430 for use in the implant generation systems too and 200 of figures la to 2 or implant adjustment and / or analysis processes 310 and 320 of figures 3b or 3c and the like. Referring to figure 4a, a cross section of a 3D model 400 of a bodily portion of a subject is illustrated that has a 3D photogrammetry mesh 401 coincident with a first 3D structural mesh 402, which is the outer surface of a first 3D structure 4023-402!), and a second 3D structural mesh 404, which is the outer surface of a second 3D structure 405 underlying the first 3D structure 4033-4036. The first 3D structure 4033-4036 may be 3 soft-tissue structure including, without limitation, for example a skin structure 403a and a soft-tissue or muscle structure 403b. The second 3D structure 405 maybe, without limitation, for example a bone structure or even another soft-tissue muscle structure overlying a bone / cartilage structure.

[00154] Referring to figure 4b, the 3d model 400 of the bodily portion of the subject may be adjusted to a desired 3D model 410 of the bodily portion of the subject as described with reference to figures la to 3c. In this example, adjustments A may be made to the 3D photogrammetry mesh 401, which in this case deforms a portion of the 3D photogrammetry mesh outwardly away from the first 3D structural mesh 402 to form a desired 3D photogrammetry mesh 411. Figure 4c illustrates the desired 3D model 420 with finalised desired 3D photogrammetry mesh 411. The desired 3D model 420 of the bodily portion of the subject includes at least data representative of: the 3D medical model including the first 3D structure 4O3a-4O3b and corresponding first 3D structural mesh 402, the second 3D structure 405 and the corresponding second 3D structural mesh 404; and the desired 3D photogrammetry data including the desired 3D photogrammetry mesh 411 and the corresponding desired texture mapping (not shown). In figure 4d, the implant analysis process as described with reference to figures la to 3c may be performed (e.g. finite element analysis or spring-mass analysis, or any other analysis / ML technique) that determines a deformation of the second 3D structural mesh 404 that causes the first 3D structures 4O3a-4O3b and the first 3D structural mesh 402 to be deformed, taking the properties of the first 3D structures 4O3a-4O3b into account, to result in a deformed first 3D structural mesh 431 and deformed first 3D structures 433a and 433b, and also a deformed second 3D structural mesh 434. This deformation may be performed iteratively until a deformation of the second 3D structural mesh 434 is found that causes the deformed first 3D structural mesh 431 to align or substantially match the desired 3D photogrammetry mesh 411. Once this is achieved, the deformed second 3D structural mesh 434 is compared with the original second 3D structural mesh 404 to determine whether there are any differences in volumes therebetween. The differences between the second 3D structural meshes 434 and 404 result in volumes or geometries 435 that represent the geometry of the required implants that should be fitted to the bodily portion of the subject during reconstructive surgery to achieve the desired outcome in relation to the desired 3D model 420.

[00155] Although the first and second 3D structures have been described herein with reference to soft-tissues and bone structures, respectively, this is byway of example only and the invention is not so limited, it is to be appreciated by the skilled person that the first and second 3D structures may both be soft-tissue structures such as when implants are required for calf or bicep body portions of the subject, or where implants are required to be placed between two soft tissue structures, as in chest implants and / or pectoral implants. Thus, the soft-tissue of the second 3D structure may be deformed when designing the implant. As well, the properties of the soft-tissue of the second 3D structure and / or proposed implant material may need to be taken into account or modelled when performing the implant analysis process. Alternatively, the first 3D structure may be a soft-tissue structure such as skin, fat, glands and / or muscletissue and the second 3D structure may be a combination of soft-tissue and / or bone / cartilage structure, for example, in relation to breast implants. Should the second 3D structure include multiple structures such as soft-tissue and bone structures, then the properties of the soft-tissue structures and / also properties of the implant may also need to be taken into account or modelled during the implant analysis process (e.g. breast implants may be silicon based fluid implants). In this case, the upper layers or soft-tissue layers of the second 3D structure maybe adjusted and the properties modelled (and / or the properties of the implants modelled) during the implant analysis process to determine suitable geometries for the implants.

[00156] As described herein, the implant material may be any type of implant material such as, without limitation, for example fluid / semi solid implant materials, soft tissue implant materials, silicon based implant material, rigid / non-rigid implant materials, rigid implant materials and the like. Given this, the above implant analysis processes may be modified to take into account the implant material properties when adjusting the second 3D structure or second 3D structural mesh, where both the first and second 3D structures (and / or implant materials within the adjusted portions of the second 3D structures) are modelled to determine the overall deformation of the first 3D structure and mesh, and determine whether the first 3D mesh aligns with the desired 3D photogrammetry mesh and the like.

[00157] Further modifications to the implant analysis process may include providing indications of the locations of each of the one or more volumes and hence the required locations of the resulting implants. When the implant material is a liquid, fluid or medium such as, without limitation, for example collagen filler, Hyaluronic Acid (HA) fillers, and / or any other type of filler or filler medium and the like, the implant analysis process may output estimated volumes that may assist in determining the volume or amount of implant material (e.g., collagen filler, HA filler or liquid implant fluid and the like) that is to be injected into the subject and also provide an indication of the injection locations and / or depths for placement of the associated implant material (e.g., collagen filler, HA filler or liquid implant fluid and the like).

[00158] The implant adjustment and analysis processes as described with reference to figures la to qd may also be applied or used during reconstructive or cosmetic surgery, where modifications may be made to implants and prior to inserting the modifications maybe scanned in 3D and placed into the 3D model of the subject to determine whether the first 3D structural mesh still aligns or coincides with the desired 3D model of the subject and / or whether the overall shape of the desired bodily portion of the subject is kept or met. This may be used during surgery to preview what various modifications to the implant will do to the desired 3D model of the subject.

[00159] Figure 5 illustrates a schematic example of a computing system / apparatus 500 for performing any of the methods described herein. The computing system / apparatus shown is an example of a computing device. It will be appreciated by the skilled person that other types of computing devices / systems may alternatively be used to implement the methods described herein, such as a distributed computing system.

[00160] The apparatus (or system) 500 comprises one or more processors 502 (e.g. CPUs). The one or more processors 502 control operation of other components of the system / apparatus 500. The one or more processors 502 may, for example, comprise a general-purpose processor. The one or more processors 502 may be a single core device or a multiple core device. The one or more processors 502 may comprise a Central Processing Unit (CPU) or a graphical processing unit (GPU). Alternatively, the one or more processors 502 may comprise specialized processing hardware, for instance a RISC processor or programmable hardware with embedded firmware. Multiple processors maybe included.

[00161] The system / apparatus comprises a working or volatile memory 504a. The one or more processors 502 may access the volatile memory 504a in order to process data and may control the storage of data in memory. The volatile memory 504a may comprise RAM of any type, for example, Static RAM (SRAM), Dynamic RAM (DRAM), or it may comprise Flash memory, such as an SD-Card.

[00162] The system / apparatus comprises a non-volatile memory 504b. The nonvolatile memory 504b stores a set of operation instructions such as operating system instructions 505a and / or implant generation instructions and the like 505b for controlling the operation of the processors 502 in the form of computer readable instructions. The non-volatile memory 504b maybe a memory of any kind such as a Read Only Memory (ROM), a Flash memory or a magnetic drive memory.

[00163] The one or more processors 502 are configured to execute operating instructions 5053-5055 to cause the system / apparatus to perform any of the methods or processes described herein. The operating instructions 5O5a-5O5b may comprise code (i.e. drivers) relating to the hardware components of the system / apparatus 500, as well as code relating to the basic operation of the system / apparatus 500. Generally speaking, the one or more processors 502 execute one or more instructions of the operating instructions 5O5a-5O5b. which are stored permanently or semi-permanently in the non-volatile memory 504, using the volatile memory 504a to store temporarily data generated during execution of said operating instructions 5O5a-5O5b.

[00164] The computer system 500 may comprise one or more network / apparatus interfaces 506 for connection to a network / apparatus, e.g. a transceiver unit which may be wired or wireless. The network / apparatus interface 506 may also operate as a connection to other apparatus such as device / apparatus which is not network side apparatus. Thus, direct connection between devices / apparatus without network participation is possible. As an option, the computer system 500 may include a user input 510 and a display 508 connected or coupled to one or more of the processors 502.

[00165] In some example embodiments, the computer system 500 may also be associated with external software applications implementing one or more of the methods and / or processes described herein. These may be applications stored on a remote server device / apparatus or one or more remote servers / apparatus such as a cloud-based platform / cloud computing platform and may run partly or exclusively on the remote server device / apparatus and the like. These applications maybe termed cloud-hosted applications. The computer system 500 maybe in communication with the remote server devices / apparatus in order to utilize the software applications stored there when implementing the methods and / or processes as described herein.

[00166] FIG. 6 shows tangible media 600, specifically a removable memory unit 602, storing computer-readable code which when run by a computer may perform methods according to example embodiments described above. The removable memory unit 602 may be a memory stick, e.g. a USB memory stick, having internal memory 604 storing the computer-readable code. The internal memory 604 may be accessed by a computer system via a connector 606. Other forms of tangible storage media may be used. Tangible media 600 can be any device / apparatus capable of storing data / information which data / information can be exchanged between devices / apparatus / network.

[00167] Implementations of the methods or processes described herein may be realized as in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These may include computer program products (such as software stored on e.g. magnetic discs, optical disks, memoiy, Programmable Logic Devices) comprising computer readable instructions that, when executed by a computer, such as that described in relation to Figure 5, cause the computer to perform one or more of the methods described herein.

[00168] Any system feature as described herein may also be provided as a method or process feature, and vice versa. As used herein, means plus function features may be expressed alternatively in terms of their corresponding structure. In particular, method aspects may be applied to system aspects, and vice versa.

[00169] Furthermore, any, some and / or all features in one aspect can be applied to any, some and / or all features in any other aspect, in any appropriate combination. It should also be appreciated that particular combinations of the various features described and defined in any aspects of the invention can be implemented and / or supplied and / or used independently.

[00170] Although several embodiments have been shown and described, it would 5 be appreciated by those skilled in the art that changes may be made in these embodiments without departing from the principles of this disclosure, the scope of which is defined in the claims and their equivalents. 27 07 23

Claims

1. A computer-implemented method of generating one or more implants for asubject for use in cosmetic or reconstructive surgery, the method comprising:obtaining a three-dimensional, 3D, model of a bodily portion of the subject, 5 wherein the bodily portion of the subject comprises a head portion of the subject comprising at least a face facial region of the subject and a neck portion of the subject comprising at least a neck region of the subject, the 3D model comprising:3D photogrammetry data representative of the bodily portion of the subject and defining a desired external appearance of the bodily portion of the10 subject, wherein the 3D photogrammetry data of the desired bodily portioncomprises a desired 3D photogrammetry mesh; anda 3D medical model representative of the bodily portion of the subject, the 3D medical model comprising a first 3D structure and a second 3D structure underlying or supporting the first 3D structure in relation to the bodily portion15 of the subject, wherein the first 3D structure comprises a first 3D structuralmesh representing the outer surface of the first 3D structure, the second 3D structure comprises a second 3D structural mesh representing the outer surface of the second 3D structure, and wherein the first 3D structure comprises data representative of skin structure and one or more soft tissue regions associated20 with the bodily portion of the subject and the second 3D structure comprisesdata representative of one or more bone or skeletal regions supporting said skin structure and one or more soft tissue regions;superimposing the first 3D photogrammetry data onto the 3D medical model based on:25 receiving a reference 3D mesh comprising a plurality of landmarkedfeatures associated with the bodily portion of the subject;identifying the plurality of landmarked features on each of the first 3D structural mesh and the first 3D photogrammetry mesh;generating a first reference 3D mesh based on superimposing the30 reference 3D mesh onto the first 3D structural mesh using the reference andidentified landmarks of the reference 3D mesh and first 3D structural mesh;generating a second reference 3D mesh based on superimposing the reference 3D mesh onto the first 3D photogrammetiy mesh using the reference27 07 23and identified landmarks of the reference 3D mesh and first 3D photogrammetry mesh; andadjusting the second reference 3D mesh to match the first reference 3D mesh based on performing one or more mesh transformation operations;5 superimposing the first 3D photogrammetiy mesh onto the first 3Dstructural mesh based on the one or more performed mesh transformation operations; andsuperimposing the 3D photogrammetry data onto the superimposed first 3D photogrammetry mesh to form the 3D model of the bodily portion of the 10 subject;determining one or more volumes generated between the 3D photogrammetry data of the desired bodily portion of the subject and the second 3D structure taking into account the structural properties of at least the first 3D structure by:automatically adjusting the first and second 3D structural meshes whilst15 modelling the structural properties of the first 3D structure until the first 3Dstructural mesh substantially coincides with the desired 3D photogrammetry mesh; andcalculating one or more volumes generated between the adjusted second 3D structure and the original second 3D structure based on determining one or20 more volume shapes resulting from differences between the adjusted second 3Dstructure and the original second 3D structure; andoutputting data representative of the determined one or more volumes for controlling the manufacture of one or more corresponding implants for the subject.25 2. The computer-implemented method of claim 1, wherein adjusting the first andsecond 3D structures taking into account at least the properties of the first 3D structure further comprising iteratively increasing or decreasing regions of the outer surface of the second 3D structure whilst modelling the structural properties of the first 3D structure until the outer surface of the first 3D structure substantially coincides with30 the desired 3D photogrammetry data.

3. The computer-implemented method of claim 2, wherein iteratively adjustingthe second 3D structure further comprising determining the extent of the first 3D structure based on modelling the structural properties of the first 3D structure in27 07 23relation to adjustments to the outer surface of the second 3D structure until the outer surface of the first 3D structure substantially coincides with the desired 3D photogrammetry data.5 4. The computer-implemented method of any of claims 1 to 3, wherein taking intoaccount the structural properties of the first and second 3D structures further comprising one or more from the group of:performing finite element analysis on the first and second 3D structures to adjust the outer surfaces of the first and second 3D structures based on the structural10 properties of the first 3D structure and until the outer surface of the first 3D structuresubstantially coincides with the desired 3D photogrammetry data;performing mass-spring analysis on the first and second 3D structures to adjust the outer surfaces of the first and second 3D structures based on the structural properties of the first 3D structure and until the outer surface of the first 3D structure15 substantially coincides with the desired 3D photogrammetry data; andperforming volumetric analysis on the first and second 3D structures to deform the outer surfaces of the first and second 3D structures based adjustments required in relation to the desired 3D photogrammetry data.20 5. The computer-implemented method of any of claims 1 to 4, wherein adjustingthe first and second 3D structural meshes taking into account at least the properties of the first 3D structure further comprising iteratively increasing outwardly or decreasing inwardly the second 3D structural mesh whilst modelling the structural properties of the first 3D structure until the first 3D structural mesh substantially coincides with the25 desired 3D photogrammetry mesh.

6. The computer-implemented method of claim 5, wherein taking into account the structure of the first and second 3D structures further comprising performing finite element analysis or mass-spring analysis on the first and second 3D structures to adjust30 the boundaries of the first 3D structural mesh and second 3D structural mesh based on modelling the structural properties of the first 3D structure and until the first 3D structural mesh substantially coincides with the desired 3D photogrammetry mesh.27 07 237- The computer-implemented method of claims 1 to 6, wherein calculating the one or more volumes generated between the adjusted second 3D structural mesh and the original second 3D structural mesh further comprising determining one or more volume shapes resulting from differences between the adjusted second 3D structural5 mesh and the original second 3D structural mesh.

8. The computer-implemented method of any of claims 1 to 7, the method further comprising:identifying one or more implant regions of the adjusted second 3D structural10 mesh forming a boundary with the original 3D structural mesh, wherein the adjusted second 3D structural mesh within the boundary extends outwardly from the original 3D structural mesh;calculating one or more implant volume shapes for each of the identified one or more implant regions based on the volume generated in each implant region between15 the original second 3D structural mesh and the adjusted second 3D structural mesh therein;outputting data representative of one or more implant volume shapes and identified implant regions for controlling manufacture of the corresponding one or more implants in relation to the bodily portion of the subject.

209. The computer-implemented method of any of claims 1 to 8, the method further comprising:identifying one or more removal regions of the adjusted second 3D structural mesh forming a boundary with the original 3D structural mesh, wherein the adjusted25 second 3D structural mesh within the boundary extends to within the interior of theoriginal 3D structural mesh;calculating one or more removal volume shapes for each of the identified one or more removal regions based on the volume generated in each removal region between the original second 3D structural mesh and the adjusted second 3D structural mesh;30 outputting data representative of one or more removal volume shapes andidentified removal regions for controlling removal of said removal regions from the bodily part of the subject.27 07 23io. The computer-implemented method of any of claims 1 to 9, wherein the bodily portion of the subject further comprising at least one from the group of:a torso portion of the subject comprising at least a shoulder, chest, abdominal and / or pelvis region of the subject;5 a limb portion of the subject comprising at least an upper extremity or armregion including one or more of an upper arm, forearm or hand region of the subject, a lower extremity region including one or more of a hip, thigh, knee, leg, ankle or foot region of the subject.10 11. The computer-implemented method of any of claims ito 10, wherein:the second 3D structure further comprising data representative of one or more second soft tissue and / or facia regions associated with the bodily portion of the subject.

12. The computer-implemented method of any preceding claim, wherein the 3D15 medical model of the bodily portion of the subject is based on data representative of a3D medical image captured from a medical imaging apparatus capable of capturing the first and second 3D structures of the bodily portion of the subject; andwherein the 3D photogrammetry data of the bodily portion of the subject is based on imaging data captured from one or more image capturing apparatus for20 generating 3D photogrammetry data representative of the external appearance of thebodily portion of the subject.

13. The computer-implemented method of claim 12, wherein the medical imaging apparatus comprises one or more from the group of:25 ultrasound or ultrasonic scanning apparatus;Computerized Tomography, CT, scanning apparatus;Photon Counting CT scanning apparatus;Magnetic Resonance Imaging, MRI, apparatus;any other medical imaging apparatus capable of capturing the first and second 30 3D structures of the bodily portion of the subject.

14. The computer-implemented method of any of claims 1 to 13, wherein obtaining the 3D model further comprising:27 07 23receiving, from an image capturing apparatus, first 3D photogrammetry data of the bodily portion of the subject requiring the one or more implants, the first 3D photogrammetry data defining an external 3D appearance of the bodily portion of the subject;receiving, from a medical image capturing apparatus, data representative of medical imagery of the bodily portion of the subject for generating the 3D medical model of the portion of the subject, the 3D medical model comprising the first 3D structure and the second 3D structure underlying or supporting the first 3D structure in relation to the bodily portion of the subject.

15. The computer-implemented method of any of claim 14, wherein obtaining the3D model further comprising:receiving data representative of desired adjustments to the superimposed first 3D photogrammetry data in relation to the first and second 3D structures of the bodily portion of the subject;controlling adjustments to the superimposed first 3D photogrammetry mesh relative to the first 3D structure of the bodily portion of the subject until the desired outcome is reached taking into account the structural properties of at least the first 3D structure; andoutputting the 3D model comprising at least the 3D photogrammetry data defining the desired external appearance of the bodily portion of the subject and the 3D medical model.

16. The computer-implemented method of claims 14 to 15, further comprising controlling the adjustments of the superimposed first 3D photogrammetry data in relation to the properties of the first and second 3D structures of the bodily portion of the subject by:identifying one or more implantable regions based on the positioning of the first 3D structure over the second 3D structure of the bodily portion of the subject and structural properties thereof;receiving data representative of desired adjustments in relation to one or more identified implantable regions;27 07 23determining the feasibility of each desired adjustment in relation to the desired adjusted one or more implantable regions based on modelling the desired adjustments of the implantable regions in relation to the structural properties of the first and second 3D structures;5 in response to determining a desired adjustment of an implantable region isinfeasible, indicating the feasibility of the desired adjustment and / or limiting the adjustment to a feasible adjustment for the implantable region, otherwise allowing the adjustment until the desired outcome is reached.10 17. The computer-implemented method of any preceding claim, the first 3Dstructure comprising a first 3D structural mesh, the first 3D photogrammetry data of the bodily portion comprising a first 3D photogrammetry mesh, wherein the 3D photogrammetry mesh has been superimposed onto the first 3D structural mesh, and obtaining the 3D model further comprising:15 controlling adjustments to the superimposed first 3D photogrammetry meshand corresponding first 3D photogrammetry data taking into account the structural properties of at least the first 3D structure of the bodily portion of the subject until a desired outcome is reached; andgenerating the 3D model based on data representative of:20 the adjusted 3D photogrammetry mesh and 3D photogrammetry datadefining the desired external appearance of the bodily portion of the subject; andthe 3D medical model comprising data representative of the first and second 3 structures and corresponding first and second 3D structural meshes.2518. The computer-implemented method of any preceding claim, the first 3D structure comprising a first 3D structural mesh, the first 3D photogrammetry data of the bodily portion comprising a first 3D photogrammetry mesh, wherein the 3D photogrammetry mesh has been superimposed onto the first 3D structural mesh, and30 obtaining the 3D model further comprising:controlling adjustments to the superimposed first 3D photogrammetry mesh and corresponding first 3D photogrammetry data taking into account the structural properties of at least the first 3D structure of the bodily portion of the subject until a desired outcome is reached; and27 07 23generating the 3D model based on data representative of:the adjusted 3D photogrammetry mesh and 3D photogrammetry data defining the desired external appearance of the bodily portion of the subject; and5 the 3D medical model comprising data representative of the first andsecond 3 structures and corresponding first and second 3D structural meshes.

19. The computer-implemented method of any preceding claim, wherein manufacturing one or more medical implants further comprises at least one of:10 3D printing one or more medical implants using the data representative of theone or more implant volumes;controlling the manufacturing of one or more medical implants based on the output data representative of the one or more implants; andcontrolling a manufacturing process at an implant manufacturer for producing 15 one or more medical implants using the data representative of the one or more implant volumes.

20. A non-transitory tangible computer-readable medium comprising data orinstruction code, which when executed on one or more processor(s), causes at least one 20 of the one or more processor(s) to perform the steps of the computer-implemented method of any of claims 1 to 19.

21. A system for generating one or more implants for a subject for use in cosmetic or reconstructive surgery, the system comprising one or more processors, a memory 25 and a communication interface, the memory comprising instructions that, when executed by the one or more processors, cause the system to perform operations according to the computer-implemented method of any of claims 1 to 19.

22. An apparatus comprising a processor, a memory unit and a communication 30 interface, wherein the processor is connected to the memory unit and thecommunication interface, wherein processor and memoiy are configured to implement the computer-implemented method according to any of claims 1 to 19.

Images