AI-assisted surgical marker and injectable filler system for facial cosmetic surgery
By applying artificial intelligence technology in facial plastic surgery, accurate surgical marking and injection schemes are generated and three-dimensional visual models are provided, which solves the subjectivity and asymmetry of traditional manual markings, improving the accuracy of the surgery and patient satisfaction.
Patent Information
- Application Number
- CN202510322172.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2025-02-26
- Filing Date
- 2025-03-19
- Publication Date
- 2025-06-27
AI Technical Summary
During facial plastic surgery, traditional manual markings have problems such as subjectivity, asymmetry, inefficiency and poor results, and the precise placement of injectable fillers and the control of injection volume are difficult, which increases the risk of surgery. At the same time, the prior art is difficult to effectively capture and demonstrate the dynamic characteristics of surgical consultation, resulting in patient dissatisfaction and uncertainty.
Using artificial intelligence (AI) technology, through computer systems and methods, accurate surgical markers and injection plans are generated, providing three-dimensional visual models, helping surgeons and patients to accurately plan and consult before surgery, and improve the efficiency of surgery design and consultation.
It improves the accuracy and efficiency of facial plastic surgery, reduces the risk of surgery, enhances patient satisfaction and confidence, and can better match the patient's expectations of surgical results.
Smart Images
Figure CN120220988A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of aesthetic medicine surgery and artificial intelligence (hereinafter referred to as "AI"), and more particularly to an AI-assisted surgical marking and injectable filler system for facial plastic surgery. Background Art
[0002] Facial plastic surgery includes a series of interventions, including rhinoplasty, blepharoplasty, facelift (rhytidectomy), otoplasty, chin augmentation, lip augmentation, brow lift, and injectable fillers. Each surgery requires precise planning to achieve optimal aesthetic and functional results.
[0003] Surgical markings, as crucial blueprints, guide surgeons to precisely perform surgeries. Traditionally, these markings are hand-drawn and highly rely on the expertise and judgment of surgeons. However, hand markings are subjective and variable, which may lead to asymmetry, low surgical efficiency, and poor results. In addition, they are time-consuming and lack reproducibility, and may not always be fully consistent with the patient's unique anatomy or desired aesthetics.
[0004] Injectable skin fillers need to be precisely placed to enhance facial contours, reduce wrinkles, or restore volume. To achieve the desired effects and minimize invasiveness, it is necessary to accurately determine the injection sites, the exact amount of filler, and understand key anatomical structures such as bones, blood vessels, and nerves. Using low-gauge needles makes it easier to have complications such as piercing blood vessels or nerves, thus exacerbating these challenges. Differences in the professional levels of injectors and anatomical structures further increase the complexity of obtaining consistent results.
[0005] An important challenge in facial plastic surgery lies in the consultation stage. Potential patients often hesitate due to the uncertainty of the results, resulting in longer consultation times for doctors to try to explain and visualize potential effects. Current methods, such as before-and-after photo comparisons, hand sketches, or static simulations, usually cannot capture the dynamic characteristics of personalized results. This mismatch may lead to dissatisfaction, hesitation, or unrealistic expectations, affecting the patient's confidence and surgical outcomes.
[0006] The progress of AI, especially in facial recognition and medical imaging, has shown great potential in various fields. Facial recognition systems use convolutional neural networks (CNNs) to analyze facial features, contours, symmetry, and proportions with high precision from images and videos. In the medical field, facial recognition algorithms have been used to diagnose syndromes, evaluate facial symmetry, and track changes in facial structure over time. However, most current systems are good at static facial analysis and lack the dynamic, real-time adaptability and responsiveness required in surgical and injection consultations. Additionally, AI solutions customized for cosmetic applications often fail to incorporate comprehensive datasets, such as demographic variations, medical imaging, and feedback loops, to ensure customization and reliability.
[0007] Integrating AI into medical procedures has shown significant promise in various fields, including diagnosis, treatment planning, and robotic surgery. However, its application in surgical marking and dermal fillers, especially in facial plastic surgery, remains under-explored. Currently, AI solutions for surgery are typically limited to image recognition or assisting intraoperative decision-making, with little attention paid to preoperative planning tools that can aid in consultations or directly impact surgical precision. Summary of the Invention
[0008] One or more embodiments of the present invention relate to applying AI to plastic surgery, such as facial plastic surgery. One or more embodiments of the present invention include devices, systems, and methods for predicting postoperative three-dimensional visualization during consultations, generating precise surgical markings for plastic surgery, and determining injection points, injection volumes, and injection trajectories for injectable fillers such as Botulinum Toxin (registered trademark) and hyaluronic acid.
[0009] One or more embodiments of the present invention provide a novel artificial intelligence-assisted system, method, and device designed to enhance the design and consultation of plastic surgery (such as facial plastic surgery). One or more embodiments can be used in clinical practice, such as during consultations and in medical training environments. One or more embodiments can be used to assist clinical practice by automatically applying markings or protocols to a patient's face with minimal intervention from a surgeon.
[0010] In at least one embodiment, a computer system is provided, which includes: at least one computer processor; an artificial intelligence (AI) engine module including at least one artificial intelligence (AI) model; and a computer memory storing instructions, which, when executed by the at least one computer processor, cause the at least one computer processor to perform the following operations, including: receiving input data input by a user into an input module; processing the input data using the AI engine module; generating a surgical plan based on the input data using the AI engine module; generating a postoperative patient facial visualization model predicted by the AI engine module based on the input data and the surgical plan generated by the AI engine module; storing the postoperative patient facial visualization model in the computer memory; and displaying the postoperative patient facial visualization model through a visualization module.
[0011] In at least one embodiment, after the step of generating the postoperative patient facial visualization model, the at least one computer processor is configured to allow the user to modify the input data; and the at least one computer processor is configured to generate an improved postoperative patient facial visualization model in the computer memory based on the modification of the input data.
[0012] The at least one computer processor may be configured to allow the user to modify the surgical plan; and based on the user's modification of the surgical plan, generate an improved postoperative patient facial visualization model in the computer memory.
[0013] The surgical plan may include surgical markings.
[0014] The at least one computer processor may be configured to generate a facial visualization model of a preoperative patient with surgical markings generated by the AI engine module, which are superimposed on the face of the preoperative patient.
[0015] The user's modification of the surgical plan may include modification of the surgical markings.
[0016] In at least one embodiment, a computer system is provided, including: at least one computer processor; an AI engine module including at least one artificial intelligence model; a computer memory storing instructions, which, when executed by the at least one computer processor, cause the at least one computer processor to perform the following operations: receiving input data input by a user into an input module; processing the input data using the AI engine module; generating a postoperative patient facial visualization model predicted by the AI engine module based on the input data; and displaying the visualization model of the postoperative patient's face through a visualization module.
[0017] The at least one computer processor may be configured to generate a visual alternative model of the postoperative patient's face as predicted by the AI engine module based on the input data, and display the visual alternative model of the postoperative patient's face through the visualization module.
[0018] The at least one computer processor may be configured to generate more visual alternative models of the postoperative patient's face based on the input data, and display the multiple visual alternative models of the postoperative patient's face through the visualization module.
[0019] The AI engine module may be configured to generate a surgical plan based on the input data and the visual model of the postoperative patient's face.
[0020] In at least one embodiment, after the AI engine module generates the visualization of the postoperative patient's face, the at least one computer processor is configured to allow the user to modify the input data; and the at least one computer processor is configured to generate an improved visual model of the postoperative patient's face based on the user's modification of the input data.
[0021] The at least one computer processor may be configured to allow modification of the surgical plan; and generate an improved visual model of the postoperative patient's face based on the modification of the surgical plan.
[0022] The at least one computer processor is configured to create a preoperative patient face visualization model by using surgical markers, which are generated by superimposing on the preoperative patient face created by the AI engine module.
[0023] In at least one embodiment, a method is provided, including: receiving, by a computer processor, input data input by a user into an input module; processing the input data by using an AI engine module; generating, by using the AI engine module, a surgical plan based on the input data; generating, based on the input data and the surgical plan generated by the AI engine module, a visual model of the postoperative patient's face as predicted by the AI engine module; storing the visual model of the postoperative patient's face in a computer memory; and displaying the visual model of the postoperative patient's face through a visualization module.
[0024] In at least one embodiment, after the step of generating the visual model of the postoperative patient's face, the method may include: allowing the user to modify the input data; and generating, based on the modification of the input data, an improved visual model of the postoperative patient's face in the computer memory.
[0025] The method may further include: allowing the user to modify the surgical plan; and generating, based on the user's modification of the surgical plan, an improved visual model of the postoperative patient's face in the computer memory.
[0026] The surgical plan may include surgical markings.
[0027] The method may further include generating a facial visualization model of a preoperative patient with surgical markings, where the surgical markings are generated by being superimposed and displayed on the face of the preoperative patient using an AI engine module.
[0028] The method may further include generating a facial visualization model of a preoperative patient with surgical markings, where the surgical markings are generated by being superimposed and displayed on the face of the preoperative patient using an AI engine module; the user's modification of the surgical plan includes modification of the surgical markings.
[0029] In at least one embodiment of the present invention, a method is provided that may include: receiving input data input by a user into an input module on a computer processor; processing the input data using an AI engine module; generating a facial visualization model of a postoperative patient predicted by the AI engine module based on the input data; and displaying the facial visualization model of the postoperative patient through a visualization module.
[0030] The method may further include: generating an alternative facial visualization model of a postoperative patient predicted by the AI engine module based on the input data, and displaying the alternative facial visualization model of the postoperative patient through the visualization module.
[0031] The method may further include: generating more alternative facial visualization models of postoperative patients predicted by the AI engine module based on the input data; and displaying multiple alternative facial visualization models of the postoperative patient through the visualization module.
[0032] The AI engine module may be configured to generate a surgical plan based on the input data and the facial visualization model of the postoperative patient.
[0033] In at least one embodiment, after the AI engine module generates a facial visualization model of a postoperative patient, the method may include allowing the user to modify the input data; and, based on the user's modification of the input data, generating an improved facial visualization model of the postoperative patient through the visualization module.
[0034] The method may further include: allowing the user to modify the surgical plan; and generating an improved facial visualization model of the postoperative patient based on the modification of the surgical plan.
[0035] The method may further include creating a facial visualization model of a preoperative patient by using surgical markings, where the surgical markings are generated by being superimposed on the face of the preoperative patient created by the AI engine module.
[0036] After the step of generating a visual model of the postoperative patient's face, the at least one computer processor may be configured to allow a user to modify the postoperative patient's face; wherein, the at least one computer processor is configured to generate an improved surgical plan based on the modification of the postoperative patient's face.
[0037] In at least one embodiment, after the AI engine module generates a visual model of the postoperative patient's face, the at least one computer processor is configured to allow a user to modify the postoperative patient's face; wherein, the at least one computer processor is configured to generate an improved surgical plan based on the modification of the postoperative patient's face.
[0038] In at least one embodiment, after the step of generating a visual model of the postoperative patient's face, the method may include allowing a user to modify the postoperative patient's face; and generating an improved surgical plan based on the modification of the postoperative patient's face.
[0039] After the AI engine module generates a visual model of the postoperative patient's face, the method may include allowing a user to modify the postoperative patient's face; and generating an improved surgical plan based on the modification of the postoperative patient's face.
[0040] In the technical solutions described and protected by the present invention, unless otherwise specified, all operations on similar "preoperative patient's face", "postoperative patient's face" and "patient's face" refer to the processing of the corresponding models (including but not limited to visual models such as images, videos, etc. and / or numerical values, etc.) generated by the computer system (such as generated by the AI engine module) or input into the system as input data, and the essence of the operation is an information processing process or method implemented by the computer (system or device).
[0041] The present invention can be applied to preoperative consultation and planning, aiming to improve the efficiency and effect of consultation and surgical plan design for cosmetic surgery (such as facial cosmetic surgery), thereby facilitating the improvement of surgical precision, greatly matching the patient's expectations for surgical results and improving the patient's satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 is a simplified block diagram of system components, methods, and / or devices of at least one embodiment of the present invention; Figure 2A is the first part of a flowchart of a first method or process according to at least one embodiment of the present invention; Figure 2B is the second part of the flowchart of the first method; Figure 3 is a flowchart of a second method according to at least one embodiment of the present invention; Figure 4It is a flowchart of the third method according to at least one embodiment of the present invention. Detailed implementation manners
[0043] Figure 1 It is a simplified block diagram of system components, methods and / or devices according to at least one embodiment of the present invention. Figure 1 。
[0044] Block Figure 1 The components in the block include computer memory 2, visualization module 4, user computer processor 6, input module 8, artificial intelligence (hereinafter referred to as "AI") engine module 10. Optionally, server computer system 12 and marking device 22. The server computer system 12 includes one or more server computer devices 14, and each device includes one or more server computer processors 16, one or more server computer memories 18 and one or more AI engine modules 20. Each AI engine module 20 may include one or more AI engine models 21, and each AI engine module 10 may include one or more AI engine models 11.
[0045] The computer memory 2 and the server computer memory 18 may be any form of computer and / or electronic memory. Each visualization module 4, input module 8, AI engine module 10 and AI engine module 20 may include or may be computer software stored in the computer memory 2 and / or the server computer memory 18 and / or other computer memories, and be executed by the user computer processor 6 and / or the server computer processor 16.
[0046] The input module 8 may be, or may include, for example, a computer, a mobile phone or a tablet computer touch screen, a computer keyboard, a 3D scanner, a camera or other such image or video acquisition devices, and / or a computer mouse.
[0047] Figure 2A It is the first part 100a of the flowchart of the first method or process according to at least one embodiment of the present invention.
[0048] In step 102, initial data input: The input data may include at least one of the following: patient data, including the type of surgery, media such as images or videos, medical images, patient information, demographic data and medical history, and desired results. These data are input into the input module 8 for transmission to the AI engine module 10 and / or 20, and are executed by the computer processor 6 and / or 16 according to the computer program stored in the computer memory 2 and / or 18.
[0049] In step 104, surgical plan generation: The AI engine module 10 is programmed to process the input data from step 102, as well as additional data sets to analyze facial anatomy, proportions, symmetry, and other relevant metrics. Based on this analysis, computer software stored in the AI engine module 10 and / or the computer memory 2 is executed by the AI engine module 10 and / or the user computer processor 6, and the AI engine module 10 generates precise surgical markings, injection plans, or surgical plans tailored to the patient's anatomy and surgical goals.
[0050] In step 106, result visualization: Programmed by computer software stored in the visualization module 4 and / or the computer memory 2 and executed by the visualization module 4 and / or the user computer processor 6, a three-dimensional visualization model of the patient's face with superimposed surgical markings, injection plans, or surgical plans, and / or of the postoperative patient's face as predicted by the AI engine module 10 and / or the visualization module 4 is generated.
[0051] In at least one embodiment, step 106 may occur before step 104. In this embodiment, in step 106, the AI engine module 10 is programmed to process the input data from step 102, as well as additional data sets to analyze facial anatomy, proportions, symmetry, and other relevant metrics; based on this analysis, computer software stored in the visualization module 4 and / or the computer memory 2 is executed by the visualization module 4 and / or the user computer processor 6, and a three-dimensional visualization model of the patient's face and / or of the postoperative patient's face as predicted by the AI engine module 10 and / or the visualization module 4 is generated.
[0052] Subsequently, in step 104, the AI engine module 10 is programmed to process the input data from step 102, the patient face data predicted in step 106, and additional relevant data sets. Based on this analysis, computer software stored in the AI engine module 10 and / or the computer memory 2 is executed by the AI engine module 10 and / or the user computer processor 6, and the AI engine module 10 generates precise surgical markings, injection plans, or surgical plans tailored to the patient's anatomy and surgical goals to achieve the results shown on the predicted patient face in step 106.
[0053] In step 108, the plan is reviewed and modified. The user can review the results provided by the visualization module 4 in step 106; the visualization module 4 may include a display, such as a computer monitor, and / or a tablet or mobile phone display screen.
[0054] The input module 8 and / or the computer software for the input module 8 are configured to allow a user to change the input data and surgical markings, injection protocols, or surgical protocols in real time through the input module 8, and the results of the changes are configured to be dynamically regenerated and displayed by the visualization module 4.
[0055] Alternatively, one or more embodiments of the present invention allow a user to make real-time changes to the desired results displayed on the face of a postoperative patient, and the surgical markings, injection protocols, or surgical protocols are configured to be dynamically regenerated and displayed by the visualization module 4 according to the changes in the desired results.
[0056] As shown in step 110, if the user makes changes to the protocol or expected results, the program will return to step 106.
[0057] If the user intends to finalize the protocol, the next process enters node A, which points to step 112 in part 100b of the flowchart, as Figure 2B shown.
[0058] In step 112, the user can finalize the protocol by approving the surgical markings, injection protocols, or surgical protocols, for example, by inputting through the input module 8 according to, for example, the computer software stored in the input module 8 and / or in the computer memory 2.
[0059] In step 114, optionally, the input module 8 and / or the computer memory 2 are programmed with computer software to allow the user to upload postoperative results at a predetermined time interval, such as by uploading facial images and / or videos of the patient just after surgery, one week after surgery, and one to three months after surgery. Optionally, the input module 8 and / or the computer memory 2 are programmed to allow the user to evaluate the proximity of these results to the predicted values generated by the AI model provided by the AI engine module 10 and provide additional feedback to the AI engine module 10 through the input module 8. The AI engine module 10 can be programmed with computer software to utilize this feedback to further optimize the model.
[0060] Figure 3 is a flowchart 200 of a second method or process according to at least one embodiment of the present invention.
[0061] In step 202, initial data input: The input data may include at least one of the following: patient data, including type of surgery, media such as images or videos, medical images, patient information, demographic data, and medical history, and desired outcomes; these input data are input into the AI engine module 10 by a computer program stored in the computer memory 2 and executed by the user computer processor 6 using the input module 8. The AI engine module 10 and / or the input module 8 are further programmed by computer software in at least one embodiment to receive additional information inputs, including manually generated surgical markings, injection regimens, or surgical plans.
[0062] In step 204, plan review: The AI engine module 10 is programmed to process the data provided in step 202, as well as the user's manual markings, injection regimens, or surgical plans, to analyze facial anatomy, proportions, symmetry, and other relevant metrics. Based on these inputs and the analysis of the AI engine module 10, the AI engine module 10 generates recommended modifications to the manual markings or surgical plans by computer software programming.
[0063] In step 206, result visualization: The AI engine module 10 generates a three-dimensional visualization model of the patient's face with the surgical markings, injection regimens, or surgical plans input by the user through the input module 8 superimposed thereon, and / or the predicted postoperative patient's face according to the surgical markings, injection regimens, or surgical plans input by the user, and displays it to the user through the visualization module 4 according to a computer program. In addition, the patient's face with the surgical markings, injection regimens, or surgical plans determined by the AI engine module 10 according to the computer program and the recommended modifications superimposed thereon, and the predicted postoperative patient's face according to the recommended modifications, are displayed to the user by the visualization module 4 according to computer programming.
[0064] In step 208, plan review and modification are performed. The user can review the results provided by the visualization module 4 in step 206. The visualization module 4 may include a computer monitor.
[0065] The input data provided by the user through the input module 8, as well as the modifications to the surgical markings, injection regimens, or surgical plans provided by the AI engine module 10, including acceptance, change, or rejection of the AI recommended modifications, are configured to be performed in real time in at least one embodiment, and the results of the modifications are configured to be dynamically displayed using the visualization module 4.
[0066] In at least one embodiment, computer software is provided, such as for the AI engine module 10 and / or the user computer processor 6, which allows the user to modify the surgical markings, injection plan, or surgical plan using the input module 8 to change the prediction results, or to modify the prediction results using the input module 8 to change the surgical markings, injection plan, or surgical plan.
[0067] As shown in step 210, if the user has modified the plan, the program returns to step 206.
[0068] If the user intends to finalize the plan in step 210, the next step of the program proceeds to Figure 3 step 212 as shown.
[0069] In at least one embodiment, in step 212, the user can finalize the plan by providing input data to the input module 8 to approve the surgical markings, injection plan, or surgical plan, which is then processed by the user computer processor 6 and the approval indication is stored in the computer memory 2 and / or displayed on the computer monitor of the visualization module 4.
[0070] Figure 4 Flowchart 300 of a third method or process according to at least one embodiment of the present invention is shown.
[0071] In step 302, the input data can include at least one of the following: patient data, including the type of surgery, media such as images or videos, medical images, patient information, demographic data, and medical history, and the desired outcome, which are input into the input module 8 for implementation by the user computer processor 6 according to the computer programming stored in the computer memory 2 and transmitted to the AI engine module 10.
[0072] In step 304, the AI engine module 10 is programmed to process the input data from step 302, as well as additional data sets to analyze facial anatomy, proportions, symmetry, and other relevant metrics; based on this analysis, at least two different versions of the three-dimensional visualization model of the postoperative patient's face are generated as predicted by the AI engine module 10 according to the patient's desired outcome, and the at least two versions are displayed to the user. Based on factors including but not limited to the following, model training based on previous surgeries, the desired outcome input by the user, the feasibility and risk factors of the surgery partially based on the input data, the AI engine module 10 can provide some visualization alternative models of the postoperative patient's face.
[0073] In step 306, the user computer processor 6 and / or the AI engine modules 10 and / or 20 are programmed to allow the user to select one of the postoperative patient facial visualization model versions to continue; the selected version is typically the one that is most consistent with the patient's expectations, taking into account outcomes, risk factors, feasibility, recovery time, and any other such factors.
[0074] In step 308, the AI engine modules 10 and / or 20 are programmed to process the input data provided in step 302, as well as additional data sets to analyze facial anatomy, proportions, symmetry, other relevant metrics, and the version selected in step 306, to generate precise surgical markings, injection plans, or surgical plans to achieve the results shown in the selected version. Optionally, surgical markings, injection plans, or surgical plans can be generated for each version of the postoperative patient face shown prior to the user selecting one of the versions in step 304. In step 308, the AI engine modules 10 and / or 20 are programmed to generate surgical plans and store these plans in the computer memory, such as memory 2, and / or display them on the visualization module 4. The AI engine modules 10 and / or 20 are programmed to generate these surgical plans based on and / or tailored specifically to the input data and the result version selected by the user through the input module 8.
[0075] In step 310, optionally, the AI engine modules 10 and / or 20 are programmed to generate three-dimensional visualization models of the patient's face with superimposed surgical markings, injection plans, or surgical plans, and / or of the postoperative patient face, and store them in the computer memory 2 and / or display them on the visualization module 4. These three-dimensional visualization models are programmed by the AI engine modules 10 and / or 20, predicted and generated based on the surgical markings, injection plans, or surgical plans, and stored in the computer memory 2 and / or displayed to the user by the visualization module 4.
[0076] Optionally, the process may flow from step 310 to node B, which flows to Figure 2A step 108 on the partial flowchart 100a shown.
[0077] Alternatively, in step 312, one or more computer processors 6, and / or the AI engine modules 10 and / or 20 are programmed to allow the user to finalize the plan by approving the surgical markings, injection plans, or surgical plans, for example, through user input to the input module 8, according to computer software, such as stored in the input module 8, computer memory 2, and / or the AI engine modules 10 and / or 20.
[0078] One or more embodiments of the present invention include a process performed by one or more computer processors and / or the AI engine modules 10 and / or 20, including the following steps: (1) Patient data collection: Input data is input through the input module 8. The input data may include at least one of the following: required procedures and patient-specific data, such as 3D facial scans, media such as photos or videos, medical images (e.g., CT scans, MRIs, X-rays, ultrasounds, etc.), demographic data (age, gender, race, etc.), medical history including previous surgeries, allergies, medication use, and other relevant factors such as risk factors, recovery time preferences, height, weight, body fat percentage, and the desired outcome of the patient. The desired outcome may be input as a specific amount of tissue removal at a specific point, a length measurement between two points or degree of repositioning, a volume or mass to be added or subtracted at a specific location, or a description of the desired final outcome, such as facial symmetry or wrinkle reduction. For example, in blepharoplasty, the input may include the patient's desire to remove excess skin and fat from the upper eyelid. This may be input as a numerical value, such as the area or volume of skin or fat removed from a specific area, or the overall desired outcome of the surgery, such as removing double eyelids or achieving symmetry between the eyes. The input data may be preprocessed by standardizing the collected data and ensuring consistent formatting. Image processing techniques may be used to enhance, segment, and extract key elements from the input media such as images or videos.
[0079] (2) Integration of various AI models: Analyze patient-specific data, such as the patient's facial anatomy (including the distribution of skin, muscle, and fat), key metrics (such as facial proportions, symmetry, and skin elasticity), and the desired outcome, for example, through one or more AI engine modules 10 and / or 20. Other information to be analyzed may include data sets such as facial recognition of other members in the patient's demographic data, surgical plans (such as surgical markings and injection plans), previous surgical outcomes, relevant anatomical models (such as bone, blood vessel, and nerve mapping, and default outcome criteria based on the selected surgery (3) Automatically generate a surgical plan according to computer programming: For example, through AI engine modules 10 and / or 20, such as precise surgical markings, injection plans, or a customized surgical plan tailored to the patient's unique facial anatomy and their desired outcome. AI engine modules 10 and / or 20 may generate multiple alternative plans, store them in the computer memory 2, and display them on the visualization module 4 for the user to consider based on risk factors, feasibility, recovery time, and expected outcomes. For example, the patient's medical images may be part of the input data that the AI engine module 10 may consider when generating a surgical plan, by adjusting the injection position, angle, depth, and needle specifications to avoid damaging key anatomical features such as bone structures, blood vessels, or nerves, and generating alternative plans based on the user's risk tolerance or recovery time.
[0080] (4) Generate a three-dimensional visualization model of the patient's face, and / or a visualization model of the patient's face with surgical plans such as surgical marks and / or injection plans superimposed thereon, as well as a predicted postoperative visualization model of the patient's face according to surgical plans such as surgical marks and / or injection plans, for example, by the visualization module 4, and executed by one or more computer processors 6, the AI engine module 10 and / or 20 according to computer software programming. In at least one embodiment, multiple versions of the postoperative visualization model of the patient's face can be generated, trained based on the AI engine 10 and / or 20 on previous procedures, risk factors, feasibility, recovery time, patient-expected outcomes, and parts of any other such factors. In at least one embodiment, the visualization model of step (4) can be generated prior to the surgical plan.
[0081] (5) A user interface (UI), such as including the input module 8 and / or the visualization module 4, allows the user to review, adjust, and approve the AI-generated surgical plan, such as surgical marks or injection plans, and / or in combination with the AI engine module 10 and / or 20 according to computer software programming. In at least one embodiment, the user can adjust the input data or modify the surgical plan, such as surgical marks and / or injection plans, through the input module 8 according to the software programming in the computer processor 6 and / or the AI engine module 10 and / or 20 to achieve the expected result. In at least one embodiment, the user can view and adjust the alternative plans generated by the AI engine module 10 and / or 20 based on risk factors, expected outcomes, or other such factors, displayed on the visualization module 4 and / or stored in the computer memory 2.
[0082] (6) In at least one embodiment, a dynamic feedback function is provided, for example, through the input module 8, the visualization module 4, and the AI engine module 10 and / or 20, to be able to regenerate realistic results in real time on the visualization module 4 and / or in the computer memory, such as the computer memory 2, based on user changes. When the user modifies the input data or surgical marks, injection plans, or surgical plans through the visualization module 4 and / or in the computer memory 2, the system (including one or more AI engine modules 10 and / or 20) is programmed to recalculate and update the visualization model in the computer memory 2 in real time and display it on the visualization module 4 to reflect the new parameters, ensuring that the surgical plan remains accurate and adaptable and providing instant feedback on the changed input data.
[0083] (7) Once satisfied, the surgical plan is finalized and saved in the computer memory of the system, such as the computer memory 2. Optionally, it is possible to scan the user's surgical plan through a photo or video, such as manual surgical markings, for example through the input module 8, analyze the input data, for example through the AI engine module 10 and / or 20, provide a visualization model of the prediction result, for example through the visualization module 4, and propose improvement suggestions for the surgical plan (such as surgical markings), which are stored in the computer memory 2. Alternatively, the user can use the input module 8 to input the surgical plan, such as an injection plan, including location, depth, and volume, via the user interface, photo, or video, analyze the input data, provide a visualization model of the result through the visualization module 4, and propose improvement suggestions for the surgical plan (such as an injection plan) through the AI engine module 10 and / or 20.
[0084] Optionally, in at least one embodiment, any one of the input module 8, the computer processor 6, the AI engine module 10 or 20 can be programmed by computer software, allowing the postoperative results to be uploaded to the AI engine module 10 or 20 through the input module 8 at defined time intervals, such as immediately after surgery, one week after surgery, three months after surgery, to further optimize the AI model of the AI engine module 10 and / or 20. In at least one embodiment, the AI engine module 10 or 20 is programmed to record the results in the form of photos, videos, 3D facial scans, medical images, or stored numerically in the computer memory 2. In at least one embodiment, the user can rate the degree of consistency between the results and the predictions generated by the model through the programming of the AI engine module 10 and / or 20 and / or by using the input module 8, provide specific feedback on any deviations, and record in the computer memory 2 any force majeure factors that may affect the results.
[0085] (10) Optionally, in step (5) or (6), the computer processor 6 and / or the AI engine module 10 and / or 20 can be programmed by computer software to allow the user to adjust the AI-generated result of the postoperative patient's face provided by the AI engine module 10 through the input module 8, so as to regenerate on the visualization module 4 a patient face image superimposed with a surgical plan such as surgical markings or injection plans according to the programming of the computer processor 6 and / or the AI engine module 10 and / or 20. The adjustment can be made by interacting with the input module 8 to change the values related to various surgical actions and regions, or visually, through the visualization module 4, such as using a mouse on a computer or a finger on a tablet, by selecting actions, clicking and dragging facial regions or sliders. In at least one embodiment, according to the computer programming of the computer processor 6 and / or the AI engine module 10 and / or 20, the surgical plan is dynamically updated in real time in the computer memory 2 based on the user's modification of the generated result of the postoperative patient's face (in at least one embodiment, these modifications are stored in the computer memory 2). In at least one embodiment, any changes made by the user to the surgical plan or the postoperative patient's face and the resulting changes to the postoperative patient's face or surgical plan generated by the AI engine module 10 can be saved in, for example, the computer memory 2 for recall and visualization, such as for side-by-side comparison.
[0086] (11) Optionally, the system can allow multi-user collaboration. For example, between a patient and a doctor, multiple users can jointly generate, review, and modify certain parts of the system.
[0087] (12) Optionally, in at least one embodiment, the marking device 22 is configured to mark or draw a surgical plan, such as surgical markings or injection plans, on the face of an actual patient based on the surgical plan generated by the AI engine 10 and / or 20. The marking device 22 can be a robotic arm or a similar device that can automatically implement the surgical plan according to the software programming of the user computer processor 6 and / or the AI engine module 10 and / or 20, such as by guiding the use of a marker without the intervention of the user or the surgeon. Alternatively, the marking device 22 can be configured to mark by directly projecting light or a laser onto the face of the actual patient. This marking device 22 can be applied at any point during the surgical procedure after the surgical plan is generated.
[0088] One or more embodiments of the present invention can use one or more computer software modules or components, which are stored in a computer memory and executed by a computer processor. The one or more computer software modules can include (a) the AI engine module 10 and / or 20, (b) the input module 8, and (c) the visualization module 4.
[0089] (a) The AI engine module 10 and / or 20, in at least one embodiment, is programmed by computer software to: (i) Use various types of AI models to generate a surgical plan, partially based on user input information.
[0090] (ii) Use a facial recognition model to identify, compare, and extract relevant biometric data, such as dimensions and specific patient conditions.
[0091] (iii) Use a generative AI model to generate a realistic three-dimensional model of the patient's face in real time. The generative AI model can obtain information from the AI and facial recognition models to generate three-dimensional visualization models of the patient's face before surgery, with the surgical or injection plan superimposed, and the predicted postoperative patient's face.
[0092] (iv) The AI model, facial recognition system, and generative AI model are used in combination to accurately capture input data, including patient data, predict the best surgical markings, injection plans, and other surgical plans and surgical outcomes, and generate various realistic patient face models. Other data sets, such as anatomical models, best practices, and previous patient outcomes, can be used in combination with the input data or as part of the AI model training. In addition, the system allows the predicted results to be visualized in real time in response to user modifications of input values, surgeries, injections, or surgical plans, and the results desired by the user. The models used can include neural networks, such as convolutional neural networks; machine learning, such as supervised learning, unsupervised learning, and deep learning, decision trees, generative AI, regression models, classifier models, and / or other such models, algorithms, and training techniques. The models can be trained on layers of a comprehensive data set that includes anatomical models, previous surgeries and filling plans and outcomes (including media, such as images or videos of the patient at various time intervals before and after surgery), surgical markings, injection plans, and other surgical plans, expectations and satisfaction with the outcomes, genetic information, demographic information, and various other medical-related information. The models can be used independently, or in series or in parallel with other models.
[0093] (v) Optionally, one or more models of the AI engine module 10 can be stored on and trained on the server computing system 12, which is communicatively coupled to the user computer processor 6. The server computing system 12 may include one or more server computing devices 14, each of which may include a server computer processor 16, a server computer memory 18, and an AI engine module 20. The server computing system 12 can receive information from the user computer processor 6, such as through the input module 8, to run the model and communicatively couple the output to the user computer processor 6. In this way, the necessary computing power can be transferred from the user computer processor 6 to the server computing system 12.
[0094] (b) The input module 8, in at least one embodiment, is programmed by computer software to: (i) Accept the desired surgical procedure and input data, the input data including patient data, including 3D facial scans, photos, videos, medical images (e.g., CT scans, MRIs, X-rays, etc.), demographic data (age, gender, race, etc.), medical history, and other relevant factors, such as height, weight, body fat percentage, and the patient's desired outcome. The desired outcome can be input as the amount of tissue removal at a specific point, the length dimension between two points or the degree of repositioning, the volume or mass increase or decrease at a specific location, or a description of the desired final outcome.
[0095] (ii) Have the ability to integrate real-time data input, such as video scans during a consultation.
[0096] (c) The visualization module 4, in at least one embodiment, is programmed by computer software to: (i) Provide a detailed, real-time 3D visualization model of the patient's face, overlaid with surgical markings and / or injection or surgical plans generated by the AI engine module 10, and the postoperative patient's face as predicted by the AI engine module 10.
[0097] (ii) Optionally overlay other information, such as key anatomical features, such as bone structures, blood vessels, and nerves, to guide the safe execution and obtain the expected results.
[0098] (iii) Allow real-time visualization of any input changes or modifications to the proposed surgical markings, and / or injection or surgical plans, and the user's desired outcome.
[0099] Example use cases for blepharoplasty: The user can input input data including their specific patient data and demographic data of blepharoplasty through the input module 8, indicating that they hope to remove a certain amount of excess skin from the upper eyelid. Hidden functions such as eyelid skin folds are an important part of blepharoplasty, and it may be necessary to manipulate the eyelid to provide a better view during imaging or video capture in order to provide sufficient information for the AI system to provide a complete analysis.
[0100] The system, such as through the AI engine module 10, is programmed and / or configured by computer software to analyze the patient's facial images and videos, and record various detected dimensions and standards of different parts of the eye and its surroundings. For example, ptosis, symmetry or asymmetry between the eyelid and the eyebrow, the amount and thickness of excess skin, inflammation, skin lesions, and various other conditions are identified and considered. In Asians, demographic-specific features such as the upper eyelid fold may also be identified. Default values may initially be used to generate surgical marks and postoperative facial images. For example, the ratio of the tarsal platform to the distance between the upper eyelid fold and the eyebrow should be 1:1.618, that is, the golden ratio. The corneal reflex-eyelid margin distance for Caucasians should be between 3.5 and 4 mm (millimeters) and symmetric. These default dimensions may be overridden by advanced users such as surgeons during data input.
[0101] The patient facial interactive model (herein, the "interactive model" is the "visualization model", and the so-called "interactive" emphasizes the way and characteristics that allow the user to modify and regenerate the visualization model based on the user's modifications), the patient facial interactive model with surgical marks superimposed, and the postoperative patient facial interactive model predicted according to the surgical marks are generated by the AI engine module 10 according to computer programming. In at least one embodiment, the input module 8 is programmed to allow the user to change the numerical values related to the surgical plan or each individual surgical mark, and the AI engine module 10 is programmed and / or configured to regenerate the surgical marks and the expected results in real time and display them through the visualization module 4. Alternatively, the user can interactively operate the surgical marks, numerical values, and expected results through the input module 8 and / or the visualization module 4, and these contents will be regenerated in real time.
[0102] Alternatively, the AI engine module 10 can generate one or more visualization models of the postoperative patient's face based on various factors, such as the previous surgical results of similar populations, risk factors, etc. For example, the AI engine module 10 can identify that the patient is Asian and has ptosis, and generate a visualized version that corrects this problem without the patient specifically mentioning the condition or inputting a correction for it when inputting the expected results. The surgical plan can be generated before or after generating the visualization model.
[0103] Once the user is satisfied with the surgical markings and results, the plan is finalized, and an indication that the plan has been finalized is configured to be stored in a computer memory (such as computer memory 2). The surgical plan can be sent to the marking device 22, which can be used to mark or draw on the actual patient's face according to the plan generated by the AI engine. The marking device 22 can be a robotic arm or a similar device that can, for example, automatically implement the surgical plan by guiding a marker without the intervention of the user or surgeon. Alternatively, the marking device 22 can make marks by directly projecting light or a laser onto the actual patient's face.
[0104] After the surgery, additional information may also be uploaded, such as images and videos of the patient at specified time periods after the surgery. In at least one embodiment, the system and / or the input module 8 is programmed to allow the user to rate the AI results generated by the AI engine module 10, compare them with the actual results, and provide feedback on patient- and surgery-specific factors that occurred during the surgery and recovery. This information can be added to the AI engine module 10 and / or the computer memory 2 for further optimization in future use.
[0105] In at least one embodiment, the AI engine module 10 is programmed by computer software to continuously scan the patient's face via video in order to dynamically optimize the input data in response to additional information collected from the patient (such as changes in posture, expression, or physical actions like lifting the eyelid) when the user interacts with the system. This feature allows for real-time updates of the proposed surgical markings or injection plan as well as the visualization model of the expected results, improving accuracy and applicability. Additionally, the AI model generated by the AI engine module 10 is programmed by computer software to reflect the patient's face in real time, changing the displayed posture, expression, or facial orientation based on the patient's movements. The real-time applicability ensures that the system remains responsive to surgeon- or patient-driven modifications, creating a seamless, interactive surgical experience.
[0106] In at least one embodiment, the processes are not limited to facial plastic surgery. These processes can be useful for various surgical procedures, such as orthopedic or general surgery. For any such process, with reference to a specific anatomical area, the face or facial aesthetics is correspondingly replaced by the relevant anatomical area. For example, for knee replacement surgery, it can be replaced by the knee; for breast augmentation surgery, it can be replaced by the breast; for neurosurgery, it can be replaced by the head.
[0107] Although the present invention has been described with reference to specific embodiments, many changes and modifications to the invention will be apparent to those with expertise without departing from the spirit and scope of the invention. Accordingly, this patent is intended to cover all such changes and modifications that are reasonably and properly included within the scope of the technical contribution of the present invention.
Claims
1. A computer system comprising: at least one computer processor; An AI engine module containing at least one AI model; A computer memory storing instructions that, when executed by at least one computer processor, cause the at least one computer processor to perform the following operations, including: receiving input data inputted by the user into the input module; Process the input data using an AI engine module; Generate a surgical plan based on the input data using an AI engine module; Generate a postoperative patient facial visualization model as predicted by the AI engine module based on the input data and the surgical plan generated by the AI engine module; storing a post-operative patient facial visualization model in a computer memory; And display the postoperative patient facial visualization model through the visualization module.
2. The computer system of claim 1, wherein: After generating the post-operative patient facial visualization model, the at least one computer processor is configured to allow a user to modify the input data; as well as The at least one computer processor is configured to generate in a computer memory an improved post-operative patient facial visualization model based on the modification of the input data.
3. The computer system of claim 1, wherein: The at least one computer processor is configured to allow a user to modify the surgical plan; as well as The at least one computer processor is configured to generate in the computer memory an improved post-operative patient facial visualization model based on the user's modifications to the surgical plan.
4. The computer system of claim 1, wherein: The surgical plan includes surgical markings.
5. The computer system of claim 4, wherein: The at least one computer processor is configured to generate a visual model of a pre-operative patient's face with surgical markings, wherein the surgical markings are generated by overlaying and displaying on the pre-operative patient's face using an AI engine module.
6. The computer system of claim 3, wherein: The surgical plan includes surgical markings; The at least one computer processor is configured to generate a facial visualization model of a preoperative patient with surgical markings, wherein the surgical markings are displayed superimposed on the preoperative patient's face by using an AI engine module; The user's modification of the surgical plan includes modification of the surgical mark.
7. A computer system comprising: at least one computer processor; An AI engine module containing at least one AI model; A computer memory storing instructions that, when executed by the at least one computer processor, cause the at least one computer processor to perform the following operations, including: receiving input data inputted by the user into the input module; Process the input data using an AI engine module; Generating a postoperative patient facial visualization model as predicted by the AI engine module based on the input data; and The postoperative patient face visualization model is displayed through the visualization module.
8. The computer system of claim 7, wherein: The at least one computer processor is configured to generate a candidate model of post-operative patient facial visualization as predicted by the AI engine module based on the input data; and The at least one computer processor is configured to display the post-operative patient face visualization candidate model via a visualization module.
9. The computing system of claim 8, wherein: The at least one computer processor is configured to generate, based on the input data, more visual candidate models of the post-operative patient's face as predicted by the AI engine module; and The at least one computer processor is configured to display, via the visualization module, a plurality of visualization candidate models of the post-operative patient's face.
10. The computer system of claim 7, wherein: The AI engine module is configured to generate a surgical plan based on the input data and a postoperative patient facial visualization model.
11. The computer system of claim 7, wherein: After the AI engine module generates the post-operative patient facial visualization model, the at least one computer processor is configured to allow the user to modify the input data; The at least one computer processor is configured to generate an improved post-operative patient facial visualization model based on user modifications to the input data.
12. The computer system of claim 10, wherein: The at least one computer processor is configured to allow a user to modify a surgical plan; as well as The at least one computer processor is configured to generate an improved post-operative patient facial visualization model based on the modification of the surgical plan.
13. The computer system of claim 10, wherein: The surgical plan includes surgical markings.
14. The computer system of claim 13, wherein: The at least one computer processor is configured to create a pre-operative patient face visualization model by using surgical markings generated superimposed on the pre-operative patient face created by the AI engine module.
15. A method comprising: receiving input data inputted by a user into an input module at a computer processor; Process the input data using an AI engine module; Generate a surgical plan based on the input data using an AI engine module; Generate a postoperative patient facial visualization model as predicted by the AI engine module based on the input data and the surgical plan generated by the AI engine module; storing a visualization model of the post-operative patient's face in a computer memory; as well as The postoperative patient face visualization model is displayed through the visualization module.
16. The method of claim 15, wherein: After the step of generating a post-operative patient facial visualization model, allowing a user to modify the input data; as well as An improved post-operative patient facial visualization model is generated in a computer memory based on the modification of the input data.
17. The method of claim 15, further comprising: Allows users to modify surgical plans; as well as Based on the user's modifications to the surgical plan, an improved post-operative patient facial visualization model is generated in a computer memory.
18. The method of claim 15, wherein: The surgical plan includes surgical markings.
19. The method of claim 18, further comprising: A facial visualization model of a pre-operative patient is generated with surgical markings, where the surgical markings are superimposed on the pre-operative patient's face using an AI engine module.
20. The method of claim 17, wherein: The surgical plan includes surgical marks; and further includes generating a facial visualization model of a preoperative patient with surgical marks, wherein the surgical marks are superimposed on the preoperative patient's face by using an AI engine module; The user's modification of the surgical plan includes modification of the surgical mark.
21. A method comprising: receiving, on a computer processor, input data entered by a user into an input module; Process the input data using an AI engine module; Generating a postoperative patient facial visualization model as predicted by the AI engine module based on the input data; and The postoperative patient face visualization model is displayed through the visualization module.
22. The method of claim 21, further comprising: Based on the input data, generating a candidate model for visualizing the postoperative patient's face as predicted by the AI engine module; as well as The visualization module displays alternative models of the postoperative patient's face.
23. The method of claim 22, further comprising: Based on the input data, generate more visual candidate models of the postoperative patient's face as predicted by the AI engine module; as well as The visualization module displays multiple visualization candidate models of the postoperative patient's face.
24. The method of claim 21, wherein: The AI engine module is configured to generate a surgical plan based on the input data and a postoperative patient facial visualization model.
25. The method of claim 21, further comprising: After the AI engine module generates a postoperative patient facial visualization model, allowing the user to modify the input data; as well as Based on the user's modification of the input data, an improved postoperative patient facial visualization model is generated through the visualization module.
26. The method of claim 24, further comprising: Allows users to modify surgical plans; as well as An improved post-operative patient facial visualization model is generated based on the modification of the surgical plan.
27. The method of claim 24, wherein: The surgical plan includes surgical markings.
28. The method of claim 27, further comprising A pre-operative patient face visualization model is created by using surgical markers that are superimposed on the pre-operative patient face created by the AI engine module.
29. The computer system of claim 1, wherein: After the step of generating a visualization model of the post-operative patient face, the at least one computer processor is configured to allow a user to modify the post-operative patient face; as well as The at least one computer processor is configured to generate a modified surgical plan based on the post-operative modification of the patient's face.
30. The computer system of claim 10, wherein: After the AI engine module generates the post-operative patient face visualization model, the at least one computer processor is configured to allow a user to modify the post-operative patient face; The at least one computer processor is configured to generate a modified surgical plan based on the post-operative modification of the patient's face.
31. The method of claim 15, further comprising: After the step of generating a post-operative patient face visualization model, allowing a user to modify the post-operative patient face; as well as A modified surgical plan is generated based on the post-operative modification of the patient's face.
32. The method of claim 24, further comprising: After the AI engine module generates a postoperative patient face visualization model, allowing the user to modify the postoperative patient face; as well as A modified surgical plan is generated based on the post-operative modification of the patient's face.