Ultrasound, ai, and ar guided system for facial injection assistance and training

Abstract

An augmented reality–assisted ultrasound guidance system (1000) enhances safety and precision during minimally invasive soft-tissue procedures. The system comprises a handheld ultrasound probe (101) for real-time imaging, a computing device (102) executing modules for image acquisition, AI segmentation, digital twin generation, and risk guidance, and an augmented reality display (103) that projects holographic overlays of anatomical structures and injection indicators. A calibration subsystem aligns the ultrasound imaging plane using facial landmarks, and a training and simulation module enables practice with prerecorded datasets and simulated filler injections. The system employs hybrid optical–IMU tracking, facial recognition for calibration, and multimodal feedback (visual, auditory, haptic). End-to-end encryption ensures secure data transmission. The risk and guidance module identifies patient-specific safe injection zones using digital twin data and assigns confidence scores, supporting both training and live guidance modes.

Description

DescriptionTitle of Invention : Ultrasound, Al, and AR Guided System for Facial Injection Assistance and Training^Technical Field

[0001] The disclosure describes an Augmented Reality (AR)-Assisted Ultrasound Guidance System. This system is fundamentally about improving spatial awareness and reducing cognitive load for the clinician.Background Art

[0002] Minimally invasive cosmetic procedures, such as facial filler and Botox injections, have become increasingly popular. However, these procedures carry risks when injectors inadvertently puncture or compress critical facial vessels or nerves. Intravascular filler injections, for example, can cause skin necrosis or even blindness if filler enters an artery. Traditionally, practitioners rely on anatomical knowledge and palpation, but individual anatomy varies and subsurface structures like arteries and nerves are not directly visible.

[0003] Ultrasound imaging has emerged as a useful tool to visualize soft tissues and vasculature in real-time during injections. High-frequency, handheld ultrasound probes can reveal the location of blood vessels and other anatomy beneath the skin, potentially allowing injectors to avoid dangerous areas. However, interpreting ultrasound images requires significant training and skill. The practitioner must coordinate between looking at a 2D ultrasound screen and the 3D patient’s face, which can be cognitively demanding and may interrupt the flow of the procedure. Indeed, portable ultrasound systems often have a separate display, forcing the practitioner to split attention between the patient and the screen. Maintaining a sterile field while manipulating an ultrasound and viewing a screen can also be cumbersome.

[0004] Augmented reality (AR) technology offers a way to keep the practitioner’s eyes on the patient by projecting relevant information into their field of view. Prior systems have explored combining ultrasound with AR to improve guidance. For example, US20210128265A1 (Jin et al.) discloses a system in which a wireless ultrasound probe streams real-time images to a smartphone and AR headset, allowing ultrasound images to be overlaid at a 1 :1 scale onto the correspondinglocation on the patient’s body. This means the ultrasound image appears in situ on the patient through the AR glasses, aligned anatomically so the clinician sees the internal view exactly where it is on the body. Similarly, US20200187901 A1 (University of California) describes an enhanced ultrasound system with an AR display: a tracked ultrasound probe sends its position / orientation to AR glasses, so the ultrasound slice can be shown in the operator’s view at the probe’s location and dynamically move with the probe. This system can even overlay a virtual instrument path - for instance, it identifies a target inside the body and projects a suggested entry point and trajectory for a needle on the AR display[6]. These developments keep the clinician’s attention on the patient and can guide tool placement.

[0005] Another relevant advance is the use of artificial intelligence (Al) to assist in ultrasound interpretation. US20220361840A1 teaches using a neural network to automatically identify blood vessels (veins and arteries) in ultrasound images and display them prominently on the ultrasound machine’s screen. The system can outline vessels, measure their diameters, and highlight them to help the user avoid or target certain vessels. This reduces reliance on the operator’s manual skill in spotting vessels in the ultrasound scan. However, this prior art does not integrate any augmented reality visualization or specific guidance for injections - it operates on the ultrasound machine’s display in a traditional manner.

[0006] Despite these advances individually, no existing solution combines handheld ultrasound, real-time Al anatomical segmentation, and AR visualization specifically for guiding facial aesthetic injections. Prior AR-ultrasound systems address general ultrasound guidance and eliminate the need to look away, but they assume the operator can interpret the ultrasound image or they focus on specific tasks like central line placement. They do not perform automated identification of facial anatomy or actively warn of injection risks. On the other hand, Al-based ultrasound vessel detection enhances imaging, but without AR it still requires the user to correlate the ultrasound image with the patient’s face mentally.

[0007] Furthermore, the field lacks a dedicated training or simulation mode that uses these technologies. Even skilled injectors undergo extensive training to understand 3D facial anatomy and safe injection practices. Some research effortshave experimented with AR for training or planning - for instance, mapping a patient’s arteries via magnetic resonance angiography (MRA) and visualizing them on the face with AR has been shown to be a useful way to reduce intravascular injection risk. One study created an AR application to project a patient’s specific arterial anatomy (derived from MRA imaging) onto their face, with an average alignment error of only ~0.3 mm, and found it could help practitioners avoid arteries during filler injections. However, that approach requires costly pre-procedure imaging (MRI) and does not incorporate real-time ultrasound or feedback during an actual injection. There remains a need for an integrated system that can be used in real procedures and in training settings alike, providing on-demand, real-time guidance without requiring expert ultrasound interpretation or preoperative imaging.

[0008] Accordingly, the present invention addresses these gaps. It provides a system and method that unify a handheld ultrasound imaging device, Al-driven segmentation of facial anatomy (veins, arteries, nerves, and soft tissue layers), and AR visualization (via smart glasses or similar devices) to guide cosmetic facial injections safely and effectively. The system not only alerts the practitioner to danger zones in real-time during an injection, but also can operate in a “training mode” where expert knowledge (ideal injection points, angles, depths, dosages) is overlaid for learning purposes. This invention thus enables safer injections by even less-experienced users, reduces the cognitive load on practitioners by simplifying ultrasound interpretation, and enhances training and education in facial aesthetics.Summary of Invention

[0009] The disclosure presents an augmented reality (AR)-assisted ultrasound guidance system aimed at improving precision and safety during minimally invasive procedures like cosmetic filler injections and regional anesthesia. The system combines an ultrasound probe, a computing device with Al segmentation, and an AR display to visualize subcutaneous anatomy in real time.

[0010] System Components

[0011] Hardware: Includes a handheld ultrasound probe, a computing device (smartphone, tablet, or embedded computer), and AR glasses. The probesupports advanced controls (e.g., capacitive, gesture-based) and spatial tracking via optical markers and inertial measurement units fused with SLAM for robust positioning.

[0012] Calibration: Uses multi-point facial landmarks (center of eyes, nose tip, lips) detected by AR headset cameras through facial recognition to create a stable and precise coordinate system for overlay alignment, ensuring accuracy despite patient micro-movements.

[0013] Digital Twin: During calibration, 3D ultrasound volumes and AR depth scans generate a patient-specific digital twin — a detailed 3D model of facial anatomy including vasculature and nerves, stored securely for simulation and procedural planning.

[0014] Software Features

[0015] Acquisition and Segmentation: Ultrasound frames and 3D volumes are acquired and processed by Al models (CNNs or transformers) to segment arteries, veins, and nerves with confidence scores displayed in AR. The digital twin integrates this data for high-fidelity anatomical representation.

[0016] Overlay Rendering: Segmented structures are color-coded and projected onto the AR display, dynamically adjusted for lighting and user preferences. In simulation mode, the system renders the digital twin on mannequins or virtual environments for practice.

[0017] Risk and Guidance: The system continuously assesses needle position relative to segmented anatomy, offering personalized injection site recommendations based on patient-specific anatomy rather than generic standards. Machine learning models account for anatomical variations, improving safety and precision. Simulations can test injection strategies on the digital twin before clinical use.

[0018] Data Management: Ultrasound images, injection metadata, and digital twin datasets are securely stored for audit, training, or regulatory purposes. The digital twin can be updated with new scans to track anatomical changes over time.

[0019] Modes of Operation

[0020] Training and Simulation: Using prerecorded datasets and digital twin models, practitioners can practice probe handling and injection planning on virtual or physical mannequins. The system adapts difficulty to enhance skill development and logs performance metrics.

[0021] Procedure Guidance: The workflow includes AR glasses calibration using fixed facial landmarks with visual cues for alignment quality. The digital twin is loaded and aligned with real-time ultrasound to provide simultaneous reference and live imaging. The system visualizes vessels and nerves during scanning, proposes safe injection points with confidence scores, and gives multi-sensory alerts during needle insertion. Overlays adapt dynamically to patient or probe movement, supported by predictive filtering and the digital twin’s stable 3D reference. Post-injection, ultrasound and Doppler scans verify filler placement and vessel status, with updates to the digital twin for future reference.

[0022] This system enhances clinician confidence and procedural safety by integrating patient-specific anatomy visualization through AR, Al segmentation, and a comprehensive digital twin model, enabling personalized treatment planning, real-time guidance, and realistic simulation prior to injections.Technical Problem

[0023] The augmented reality-assisted ultrasound guidance system described in present document, addresses major challenges in the cosmetic filler and soft- tissue intervention industry by providing real-time, patient-specific visualization and guidance. Traditional approaches often rely on generic anatomical standards and blind injection techniques, which increase the risk of vascular complications, such as occlusion, tissue necrosis, or blindness, due to accidental needle placement near critical arteries or veins. This invention integrates ultrasound imaging, Al-powered anatomical segmentation, and AR overlays anchored with fixed facial landmarks, delivering precise, stable visualization of patient anatomy even during micro-movements. By constructing and simulating with a digital twin — a 3D model generated from ultrasound and surface scan data — the system offers personalized mapping and risk evaluation for every patient, replaces generic standards with tailored guidance, and enables safe, accurate injection planning and execution. Multimodal feedback and continuous risk alerts further enhance safety during procedures; encrypted data handling ensures patientprivacy and regulatory compliance. The solution also introduces advanced training capabilities, allowing practitioners to rehearse patient-specific scenarios in simulation mode, improving skill and reducing adverse event risk. Collectively, these advancements overcome documented problems of poor precision, high complication rates, limited personalization, calibration drift, and training deficiencies in current practice and prior art, representing a significant leap in safety, effectiveness, and practitioner confidence in minimally invasive procedures.Advantageous Effects of Invention

[0024] The augmented reality-assisted ultrasound guidance system revolutionizes minimally invasive procedures, such as cosmetic filler injections and regional anesthesia, by integrating a handheld ultrasound probe, Al-driven anatomical segmentation, and a head-mounted AR display. This system provides clinicians with real-time, patient-specific visualization of subsurface anatomy, reducing the risk of complications like intravascular injections. Unlike prior art, it eliminates the need for advanced ultrasound expertise or prior imaging, as the Al interprets images and the AR display presents intuitive, spatially aligned overlays. The system leverages compact, commercially available hardware (e.g., wireless probes, smartphones, AR glasses) or a dedicated all-in-one headset, making it adaptable to typical clinic environments without bulky equipment. Its training mode, using patient-specific simulations, accelerates learning by visualizing anatomy and guiding safe injection techniques, shortening the learning curve compared to traditional didactic or cadaver-based training. The system’s novelty lies in merging ultrasound, Al, and AR with domain-specific knowledge for facial aesthetics, offering a unique, integrated platform for both procedural guidance and training. A digital twin, a 3D model of the patient’s facial anatomy, is generated from ultrasound and surface scans, enabling pre-procedural simulation on mannequins or virtual environments. This supports personalized training and planning, accounting for anatomical variations. Fixed facial landmarks (center of eyes, tip of nose, lips) are used for rapid AR calibration, ensuring stable overlays via facial recognition algorithms. The Al-driven risk module identifies safe injection sites tailored to individual anatomy, using a weighted risk function and a huge patient database, validated in very short time.Brief Description of Drawings

[0025] The figures are not intended to be exhaustive or limited to the precise form disclosed. It should be understood that the invention can be practiced with modification and alteration, and that it is limited only by the claims and the equivalents thereof.

[0026] [Fig.1 shows the hardware components of the system that integrates medical imaging with augmented reality.

[0027] Fig.2 depicts the architecture of an AR ultrasound guidance system, which integrates real-time ultrasound imaging and Al segmentation to provide anatomical labeling and risk assessment

[0028] Fig.3 illustrates the AR ultrasound guidance system that uses real-time ultrasound and Al segmentation to provide anatomical labeling and risk assessment.

[0029] Fig.4 shows the flowchart step of the process of the invention.]Description of Embodiments

[0030] The present disclosure relates to an augmented reality-assisted ultrasound guidance system configured to enhance safety and precision during minimally invasive procedures such as cosmetic filler injections, regional anesthesia, or other soft-tissue interventions. The system integrates an ultrasound imaging probe, a computing device executing an artificial intelligence (Al) segmentation module, and a head-mounted augmented reality (AR) display for real-time visualization of subcutaneous anatomy.

[0031] System Overview

[0032] In one embodiment, the system comprises three principal hardware components. Fig. 1 shows a diagram illustrating the hardware components of the said system that integrates medical imaging with augmented reality. The three main components depicted are:

[0033] 1. A hand held ultrasound probe(101 ), which captures real-time images.

[0034] 2. A computing device(102), which processes data and runs Al algorithms.

[0035] 3. An AR display (headset)(103), which receives processed data and projects holorengic overlays onto the user's view (likely a patient's anatomy), operatively interconnected for real-time acquisition, analysis, and visualization of ultrasound data aligned with patient anatomy.

[0036] The system further includes software modules(200) responsible for (i) image acquisition, (ii) anatomical segmentation, (iii) overlay rendering, (iv) digital twin generation, and (v) risk assessment and guidance logic. The disclosed configuration enables the practitioner to visualize patient-specific vasculature and tissue planes directly superimposed on the treatment area and to simulate procedures using a patient-specific digital twin prior to intervention. The system is designed to integrate with existing clinical workflows, supporting DICOM compatibility for data export to electronic medical records (EMRs) and adhering to FDA and CE medical device standards for regulatory compliance.

[0037] Hardware Components

[0038] Ultrasound Probe (101):

[0039] The probe may be a high-frequency linear array transducer suitable for superficial structures of the face. It may be wired to the computing device (102) or wireless with an onboard battery and transmitter. Controls such as depth adjustment, freeze, and capture functions may be located on the probe housing, optionally through physical buttons, capacitive touch sensors, or gesture-based interfaces.

[0040] In one embodiment, capacitive or gesture-based sterile controls are provided on the probe housing, enabling manipulation of ultrasound parameters without physical contact, thereby reducing contamination risks while maintaining full operability.

[0041] In one embodiment of the invention, the probe operates at 10-18 MHz for high-resolution imaging of superficial facial structures, with a penetration depth of 2-5 cm and a frame rate of 30-60 fps to ensure smooth real-time visualization. Wireless models use a 2.4 / 5 GHz Wi-Fi 6 module or Bluetooth 5.0 for low-latency data transmission (latency <20 ms), with a 3,000 mAh battery supporting 4-6 hours of continuous operation. The probe is lightweight (<300 g) andergonomically designed for prolonged use, with IPX7 water resistance for easy sterilization.

[0042] Spatial Tracking:

[0043] To establish probe orientation, the probe may carry an AR-visible marker (e.g., QR code, IR LED constellation) detected by the AR glasses’ forward camera. Alternatively, inertial measurement units (IMUs) within the probe may provide orientation, with sensor fusion algorithms correcting drift through simultaneous localization and mapping (SLAM) performed by the glasses.

[0044] In a further embodiment, hybrid optical-IMU fusion is implemented, wherein IMU data supplies high-frequency updates and optical tracking corrects accumulated drift, thereby ensuring robust tracking even under temporary occlusion or variable lighting conditions.

[0045] For example, he IMU includes a 9-axis sensor (accelerometer, gyroscope, magnetometer) with a sampling rate of 100 Hz, achieving positional accuracy of ±1 mm and angular accuracy of ±0.5°. The AR-visible marker is a 5x5 cm QR code or a 4-point IR LED array, scanned at 60 Hz by the AR headset’s 12 MP camera, supporting tracking distances of 0.3-1 .5 m. SLAM algorithms leverage a 3D point cloud with a density of 10,000 points / m2for robust environmental mapping.

[0046] Calibration:

[0047] A calibration routine aligns the ultrasound imaging plane with AR coordinate space. This may involve contacting known facial landmarks (e.g., nasal tip, chin) or a calibration phantom prior to the procedure.

[0048] In one configuration, a multi-point landmark calibration routine is performed using at least three anatomical landmarks, combined with drift correction routines, thereby ensuring millimeter-level overlay accuracy even with patient micro-movements.

[0049] Some fixed parts of the face, such as the center of the eyes, the tip of the nose, and the lips, can be considered as fixed reference points for starting and calibrating the AR. These landmarks are detected by the AR headset’s forwardfacing cameras using facial recognition algorithms, establishing a robustcoordinate system for aligning the ultrasound imaging plane with the AR display. This approach ensures consistent calibration across sessions and enhances overlay stability even with minor patient movements.

[0050] The calibration routine is completed in under 60 seconds using a semiautomated process. The practitioner places the probe on the specified landmarks (center of eyes, tip of nose, lips), guided by AR overlays showing real-time alignment feedback (e.g., crosshair deviation <1 mm). The system employs a RANSAC-based algorithm to compute a transformation matrix, achieving overlay accuracy of ±0.5 mm. A disposable calibration phantom (10x10x5 cm, with embedded echogenic markers) is provided for optional use in sterile environments, ensuring compliance with clinical hygiene standards.

[0051] Digital Twin Integration:

[0052] The ultrasound probe may also be used to acquire a 3D volumetric dataset of the patient’s facial anatomy during the calibration phase. This dataset, combined with surface scans from the AR headset’s depth-sensing cameras, enables the construction of a patient-specific digital twin, a 3D digital model replicating the patient’s facial geometry and subsurface anatomy. This model is stored on the computing device and used for pre-procedural simulation and planning.

[0053] The 3D ultrasound dataset is acquired at a resolution of 0.1 mm3 / voxel, with a scan time of 2-3 minutes. Surface scans are performed using a time-of-flight (ToF) camera with a depth resolution of 0.2 mm at 30 fps, covering a 30x30 cm field of view. The digital twin model is generated in under 5 minutes, with a file size of -500 MB, and is compatible with standard 3D rendering formats (e.g., OBJ, STL) for interoperability with other medical imaging systems.

[0054] Computing Device (102):

[0055] The computing device may be a smartphone, tablet, wearable computer, or embedded module within the AR headset. It executes the Al segmentation pipeline, digital twin generation algorithms, manages communication between components, and handles secure data storage. Wireless probes transmit via Bluetooth, Wi-Fi, or proprietary protocols.

[0056] (In one embodiment, the computing device implements end-to-end encryption using TLS or DTLS protocols for all ultrasound data transmissions, includingdigital twin datasets, thereby ensuring patient data privacy and compliance with medical cybersecurity standards.

[0057] The computing device is equipped with a minimum of 8 GB RAM, a 2.5 GHz octa-core processor, and a dedicated GPU (e.g., NVIDIA Jetson Nano or equivalent) to support real-time Al processing and 3D rendering at 30 fps. It runs a Linux-based OS with a medical-grade software stack, supporting DICOM export and HL7 integration for seamless EMR connectivity. Data storage uses a 256 GB SSD with AES-256 encryption, and the device supports 5G connectivity for cloudbased backups or remote consultations.

[0058] Digital Twin Processing:

[0059] The computing device processes ultrasound and surface scan data to generate a digital twin using 3D reconstruction algorithms, such as marching cubes or voxel-based modeling. The digital twin integrates ultrasound-derived subsurface structures (e.g., vasculature, nerves) with surface geometry from depth-sensing cameras, creating a high-fidelity model for simulation. The model is stored in a secure, patient-specific database and can be updated with additional scans to reflect anatomical changes over time.))

[0060] (The marching cubes algorithm generates a polygonal mesh with a vertex density of 50,000-100,000 triangles, optimized for real-time rendering. The digital twin includes a layered anatomical model (skin, fat, muscle, vasculature, nerves) with a spatial resolution of 0.2 mm. Updates to the model are processed incrementally, requiring ~30 seconds per scan, and are versioned to track anatomical changes over multiple procedures.

[0061] Fig.2 displays the system's architecture, demonstrating the interconnections and data flow among its primary components:

[0062] 1. Hardware Components(I OO): Including an Ultrasound Probe, Computing Device, and AR Display (Headset).

[0063] 2. Software modules(200): Such as image acquisition, Al segmentation, overlay rendering, risk & guidance, and data management, which process the Ultrasound Data.

[0064] 3. Digital twin & planning(300): Which uses AR display overlay data to create a patient-specific 3D model and enable Pre-procedional simulation.

[0065] 4. Operating Modes: The system functions are divided into a training mode (using simulated data for skill development) and a live guidance mode (covering calibration, planning, injection execution, and post-procedure verification).

[0066] AR Display (103):

[0067] The AR device may be optical see-through glasses, allowing visualization of the patient with holographic overlays such as vessel outlines, injection markers, or safety alerts. Devices may include built-in cameras or rely on external sensors for spatial mapping.

[0068] In one embodiment, the AR headset incorporates depth-sensing modules, such as structured-light or time-of-flight sensors, thereby reducing drift and anchoring overlays with improved spatial accuracy.

[0069] The AR display may further provide multimodal output: visual (color-coded overlays), auditory (alerts, guidance prompts), and haptic (probe vibration, wristband pulses).

[0070] In one configuration, a graduated multimodal feedback system is implemented, wherein low-severity warnings are conveyed by subtle tactile pulses, while high-severity risks are conveyed by combined haptic, auditory, and visual alerts.

[0071] The AR display can also render the digital twin in a simulation mode, allowing practitioners to visualize and interact with the patient-specific 3D model overlaid on a physical mannequin or in a fully virtual environment.

[0072] The AR headset features a 1920x1080 resolution per eye, a 90° field of view, and a refresh rate of 90 Hz for smooth holographic rendering. It includes a 6 DoF (degrees of freedom) tracking system with <1 ms latency, powered by a 4,000 mAh battery for 3-4 hours of continuous use. The headset supports Wi-Fi 6 and Bluetooth 5.0 for data streaming and is compatible with prescription lens inserts for practitioner comfort. Haptic feedback is delivered via a 50 Hz vibration motor in the probe or a paired wristband, with customizable intensity levels.

[0073] Software Components

[0074] Image Acquisition Module:

[0075] Interfaces with the ultrasound probe to stream frames to the computing device. Pre-processing may include noise reduction, speckle filtering, and frame stabilization.

[0076] The module also supports the acquisition of 3D ultrasound volumes for digital twin construction, ensuring high-resolution data capture for accurate anatomical modeling.

[0077] Pre-processing includes a Gaussian filter (o=1 .5) for speckle reduction and a frame stabilization algorithm with a temporal smoothing window of 5 frames, achieving a signal-to-noise ratio (SNR) improvement of 10 dB. The module supports a frame buffer of 1 ,000 frames, with real-time compression (H.264) to reduce bandwidth to <50 Mbps for wireless transmission.

[0078] Al Segmentation Engine:

[0079] Trained convolutional neural networks (CNNs) or transformer-based models identify critical structures such as arteries, veins, and nerves within each frame. Detected structures are assigned confidence values and categorized by risk level.

[0080] In one embodiment, the system displays a confidence indicator within the AR overlay, such as “artery detected - 85% confidence,” thereby allowing the clinician to weigh Al recommendations against clinical expertise.

[0081] The Al segmentation engine also processes 3D ultrasound data to map subsurface structures into the digital twin, ensuring that vessels, nerves, and other critical anatomy are accurately represented for simulation and planning.

[0082] The Al model is a U-Net-based CNN or Vision Transformer, trained on a dataset of 10,000 annotated ultrasound images, achieving a Dice coefficient of 0.92 for vessel segmentation and 0.89 for nerve segmentation. Inference runs at 20 ms per frame on the computing device’s GPU, with a false positive rate of <5% for critical structures. The model is optimized for edge deployment using TensorRT, reducing memory usage to <2 GB.

[0083] Overlay Rendering Engine:

[0084] Projects segmented structures onto the AR display. A vein may appear as a blue line, an artery as red, and a nerve as yellow. The overlay is anchored to the patient’s skin surface, updating dynamically with probe motion.

[0085] In one configuration, adaptive transparency blending is applied to the AR overlays, wherein opacity automatically adjusts according to ambient brightness or user preference, thereby preventing obstruction of surface anatomy while maintaining visibility of subsurface structures.

[0086] In simulation mode, the overlay rendering engine projects the digital twin’s anatomical structures onto a mannequin or virtual environment, enabling practitioners to visualize and interact with the patient-specific model during preprocedural practice.

[0087] The rendering engine uses OpenGL for real-time overlay projection, supporting 10,000 polygons per frame at 60 fps. Adaptive transparency is controlled by a luminance-based algorithm (threshold: 100-1 ,000 lux), with user- adjustable opacity sliders accessible via voice commands or the AR headset’s touchpad. Overlays are depth-registered to the patient’s skin surface with a reprojection error of <0.3 mm.

[0088] Risk and Guidance Module:

[0089] Continuously evaluates probe position, needle trajectory (if visible), and anatomical segmentation. Provides advisory injection sites, needle entry angles, and insertion depths. Issues warnings when the needle tip approaches high-risk structures.

[0090] In one embodiment, vessel-specific alert thresholds are applied, with stricter proximity limits for arteries and more relaxed limits for superficial veins, thereby improving safety while minimizing false alarms.

[0091] The algorithm can determine safe injection areas based on each person’s specific anatomy, not just a generic standard. By analyzing the patient-specific digital twin and real-time ultrasound data, the module identifies optimal injection sites tailored to the individual’s unique vasculature and tissue planes, reducing the risk of complications. The system employs machine learning models trained on diverse anatomical datasets to adapt recommendations to variations in facial anatomy, such as vessel depth, size, or atypical branching patterns.

[0092] The module also supports simulation scenarios using the digital twin, allowing practitioners to test injection strategies and evaluate risk profiles in a virtual environment before performing the procedure on the patient.

[0093] The module uses a proximity-based risk model, with thresholds of 2 mm for arteries, 5 mm for veins, and 3 mm for nerves, calculated using Euclidean distances in 3D space. Safe injection sites are scored using a weighted risk function (e.g., 70% vessel proximity, 20% tissue depth, 10% anatomical variability), with a processing latency of <50 ms. The system integrates with a needle tracking algorithm (accuracy: ±0.5 mm) using echogenic feature detection, and alerts are triggered with a response time of <100 ms to ensure real-time safety.

[0094] Data Management:

[0095] Records ultrasound images, injection coordinates, ((digital twin datasets,)) and procedural metadata for training, audit, or regulatory purposes.

[0096] The digital twin is stored in a secure, patient-specific format, such as DICOM or a proprietary 3D model format, and can be retrieved for future procedures or updated with new scans to maintain accuracy.

[0097] Data is stored in a HIPAA-compliant cloud database with redundant backups, supporting up to 1 TB of storage per clinic. Metadata includes timestamped injection coordinates (x, y, z), filler volume (0.1 mL resolution), and procedure duration. The system generates automated PDF reports summarizing procedure details, compatible with EMR systems via HL7-FHIR protocols.

[0098] Training and Simulation Mode

[0099] A training mode may use prerecorded ultrasound datasets with known vessel annotations. The AR glasses project simulated overlays onto a mannequin face or virtual model. Practitioners can practice probe placement, structure identification, and injection planning without involving live patients.

[0100] In one embodiment, the training module applies adaptive difficulty scaling, wherein successive training sessions present smaller vessels, lower contrast, or more ambiguous anatomy, thereby improving user skill progression.

[0101] Digital Twin Simulation:

[0102] A digital model (Digital Twin) of each patient’s face can be created to practice and simulate before injection. The digital twin is generated by combining 3D ultrasound volumes with surface scans from the AR headset’s depth-sensing cameras, creating a high-fidelity 3D model of the patient’s facial anatomy, including skin surface, vasculature, nerves, and other subsurface structures. In simulation mode, the AR display renders the digital twin overlaid on a physical mannequin or within a fully virtual environment. Practitioners can manipulate a virtual ultrasound probe and needle to practice injection techniques, with the Al segmentation engine providing real-time feedback on structure identification and risk assessment. The digital twin supports multiple simulation scenarios, such as varying injection sites, depths, or filler volumes, and can incorporate patientspecific anatomical variations, such as atypical vessel patterns. The system logs simulation outcomes for performance analysis and skill improvement, with metrics such as injection accuracy and avoidance of high-risk structures.

[0103] The training mode includes a library of 50 preloaded patient-specific digital twins, each with annotated anatomical variations, accessible offline for practice. Simulations run at 30 fps with a latency of <50 ms, and performance metrics include success rate (target site hit rate: >95%), time to completion (target: <5 minutes), and error rate (false positives: <3%). The module supports multi-user sessions via a cloud-based training portal, allowing remote instructor feedback with a latency of <200 ms.

[0104] Simulation of Filler Volume Changes:

[0105] In training mode, the digital twin is configured to simulate the distribution of injectable materials, such as dermal fillers. For example, when the user selects and injects 1 mL or 2 mL of filler into a predefined site on the digital twin, the system dynamically updates the 3D facial model to show changes in contour, tissue volume, and potential vascular compression. This real-time simulation helps practitioners understand the aesthetic and anatomical effects of different injection volumes, enhancing learning of dosage control and risk management. Visual overlays in AR may show before / after comparisons or volumetric heatmaps to demonstrate filler spread and tissue displacement.

[0106] Procedure Guidance Mode

[0107] 1. Setup:

[0108] Practitioner dons AR glasses, links the probe, and performs calibration. A facial landmark crosshair may be overlaid to confirm alignment.

[0109] In one configuration, a guided auto-calibration workflow is presented, including real-time alignment scoring (e.g., green / yellow / red indicator), thereby confirming calibration quality before clinical use.

[0110] Some fixed parts of the face, such as the center of the eyes, the tip of the nose, and the lips, are used as fixed reference points for AR calibration. The AR headset’s facial recognition algorithms automatically detect these landmarks to initialize the AR coordinate system, ensuring precise alignment of ultrasound overlays with the patient’s anatomy. A guided calibration interface provides visual feedback to confirm accurate landmark detection before proceeding.

[0111] The setup phase may also include loading the patient’s digital twin, which is aligned with the real-time ultrasound and AR coordinate space to ensure consistency between simulation and live procedure.

[0112] Setup is completed in a sterile field using a touchless interface (voice or gesture-based), with probe pairing via NFC or QR code scanning in <10 seconds. The calibration interface displays a progress bar and audible confirmation tones (1 kHz, 70 dB), with a success threshold of <0.5 mm alignment error. The digital twin is loaded from the cloud or local storage in <15 seconds, with automatic verification of patient ID via barcode or RFID tag to prevent mismatches.

[0113] 2. Scanning:

[0114] Probe is placed on target anatomy. AR glasses show live ultrasound slices superimposed on the face, with Al-highlighted vessels and nerves.

[0115] In one embodiment, dynamic transparency blending of ultrasound slabs is applied, automatically adjusting overlay opacity to maintain surface visibility under varying lighting conditions.

[0116] The digital twin’s anatomical data can be displayed alongside live ultrasound images, providing a reference for expected subsurface structures and enhancing confidence in real-time scanning.

[0117] The ultrasound feed is displayed at a resolution of 1280x720, with a depth range of 1-5 cm, adjustable in 0.5 cm increments via voice commands. The AR overlay updates at 60 Hz, with a jitter of <0.2 mm, and supports zoom levels of 1x-4x for detailed inspection. The digital twin’s reference structures are toggled on / off via a hand gesture, ensuring minimal disruption to the clinical workflow.

[0118] 3. Planning and Guidance:

[0119] If injection point coincides with a vessel, the system suggests an alternate safe site (green dot) with trajectory arrows and depth annotations.

[0120] In one embodiment, each suggested injection site is accompanied by a displayed confidence score, enabling the clinician to assess Al reliability in real time.

[0121] The algorithm determines safe injection areas based on the patient’s specific anatomy, leveraging the digital twin and real-time ultrasound data to propose sites tailored to the individual’s unique vasculature and tissue planes. This personalized approach accounts for anatomical variations, ensuring safer and more effective injection planning.

[0122] The system cross-references the planned injection site with the digital twin’s anatomical model to validate safety and optimize trajectory, drawing on simulation data to recommend sites previously tested in the virtual environment.

[0123] Safe injection sites are visualized as 3D holographic markers (diameter: 5 mm) with trajectory arrows (length: 10-20 mm) and depth annotations (resolution: 0.1 mm). The system calculates up to 5 alternative sites per region, ranked by a safety score (0-100), with a processing time of <100 ms. Recommendations are validated against a database of 1 ,000+ patient anatomies, ensuring robustness across diverse facial structures.

[0124] 4. Injection Execution:

[0125] As the needle advances, the Al detects its echogenic tip. Proximity to a vessel triggers risk alerts (flashing overlay, beep, vibration). Injection volume counters may also be displayed.

[0126] In one embodiment, vessel-specific proximity thresholds are configured, such that warnings for critical arteries trigger at greater distances than for minor superficial veins, thereby enhancing specificity of alerts.

[0127] The digital twin’s anatomical map supports needle tracking by providing a 3D reference for expected vessel locations, improving the accuracy of proximity alerts during injection.

[0128] Needle tracking achieves a detection accuracy of ±0.3 mm using a gradientbased edge detection algorithm, with alerts triggered via a 500 ms flashing red overlay (RGB: 255,0,0), a 2 kHz beep (80 dB), and a 100 Hz haptic pulse. Injection volume is tracked with a resolution of 0.01 mL, displayed as a numeric overlay (font size: 12 pt). The system supports needles of 25-30 gauge, with automatic calibration for echogenicity.

[0129] 5. Dynamic Updates:

[0130] Overlays remain anchored despite minor patient or probe motion. Predictive algorithms maintain stable visualization during transient occlusions.

[0131] In one configuration, predictive tracking models, such as Kalman filtering, are employed to stabilize overlays, reducing jitter and ensuring smooth visualizations during probe or patient motion.

[0132] The digital twin’s 3D model enhances dynamic updates by providing a stable reference for anatomical structures, ensuring consistent overlay alignment even with patient micro-movements.

[0133] The Kalman filter processes 6 DoF motion data at 100 Hz, reducing overlay jitter to <0.1 mm. The system compensates for patient movements up to 5 mm / s and probe rotations up to 10° / s, with a re-anchoring latency of <50 ms. Transient occlusions are handled by buffering 10 seconds of tracking data, ensuring seamless visualization.

[0134] 6. Completion:

[0135] Post-injection, ultrasound verifies filler placement and vessel patency. Optional Doppler overlays assess perfusion. Session data are logged for recordkeeping.

[0136] In one embodiment, Doppler-based perfusion overlays are displayed within the AR view, enabling immediate confirmation of vascular safety following filler injection.

[0137] The digital twin is updated with post-procedure ultrasound data to reflect changes in anatomy, such as filler distribution, and stored for future reference or comparison in subsequent procedures.

[0138] Doppler overlays operate at a pulse repetition frequency of 2-5 kHz, with a velocity resolution of 1 cm / s, displayed as a color-coded map (red / blue for flow direction). Post-procedure scans are completed in <30 seconds, with filler placement verified against a 3D density map (resolution: 0.1 mm3). Session data are exported to EMRs within 5 seconds via a secure API, with a digital signature for auditability.

[0139] The Fig.3 displays a detailed block diagram outlining the workflow and components of an AR Ultrasound Guidance System.

[0140] The system integrates Image Acquisition (via an ultrasound scanner) and Al Segmentation (for real-time anatomical labeling) as its primary inputs. The core guidance system manages and distributes data to various functions:

[0141] 1 . Risk Assessment: Including Risk & Rendering (for injection safety evaluation and high-risk alerts visualized via an AR overlay) and Risk & Guidance (for optimal site suggestion).

[0142] 2. Data Management: Storing procedural data, images, digital twin models, and ensuring regulatory compliance.

[0143] Finally, the system supports two operational outputs: a Training Mode for skill development using simulated data, and a Live Guidance Mode for clinical procedures involving calibration, planning, injection execution, and postprocedure verification.

[0144] The above embodiments as described are only illustrative, and not intended to limit the technique approaches of the present invention. Although the present invention is described in details referring to the preferable embodiments, those skilled in the art will understand that the technique approaches of the present invention can be modified or equally displaced without departing from theprotective scope of the claims of the present invention. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. Any reference signs in the claims should not be construed as limiting the scope.Industrial Applicability

[0145] The system is designed for practical implementation in clinical settings, particularly in cosmetic surgery, dermatology, anesthesiology, and interventional radiology. Its technical specifications and workflow integration ensure applicability across diverse medical environments, enhancing procedural safety, efficiency, and trainings

Claims

Claims

1. An Ultrasound, Al, and AR Guided System for Facial Injection Assistance and Training, comprising:(a) a handheld ultrasound probe (101 ) configured to acquire real-time ultrasound images of subcutaneous tissue;(b) a computing device (102) operatively connected to the ultrasound probe (101 ), the computing device including:(i) an image acquisition module configured to receive ultrasound image frames and perform pre-processing including speckle noise reduction and frame stabilization;(ii) an Al segmentation engine trained to detect and label anatomical structures including vessels and nerves within each image frame;(iii) a digital twin generation module configured to construct a patient-specific three-dimensional model of facial anatomy by combining ultrasound volumetric data with surface scans acquired by an augmented reality display; and(iv) a risk and guidance module configured to evaluate probe position, needle trajectory, and anatomical segmentation to determine safe injection sites and issue risk alerts;(c) an augmented reality display (103) configured to project holographic overlays aligned with the patient’s anatomy, the overlays comprising segmented anatomical structures and injection guidance indicators; and(d) a calibration subsystem configured to align the ultrasound imaging plane with an augmented reality coordinate space based on at least three facial landmarks, e) a training and simulation module configured to display prerecorded ultrasound datasets on a virtual model and simulate filler injections with real-time updates to tissue contours and volume; wherein the system is configured to operate in a training mode using simulated datasets and a live guidance mode during clinical procedures, and to update overlay visualization dynamically in response to probe or patient movement.

2. The system of claim 1 , wherein the ultrasound probe (101 ) includes an inertial measurement unit (IMU) and an optical tracking marker, and the computing device (102) performs hybrid optical-IMU fusion for orientation tracking and drift correction.

3. The system of claim 1 , wherein the calibration subsystem identifies facial landmarks including the nasal tip, eyes, and lips using facial recognition algorithms executed on the augmented reality display (103).

4. The system of claim 1 , wherein the digital twin integrates both ultrasound- derived subsurface data and surface geometry from a depth-sensing camera.

5. The system of claim 1 , wherein the augmented reality display (103) provides multimodal feedback including:(a) visual overlays color-coded by anatomical structure type,(b) auditory alerts for proximity warnings, and(c) haptic feedback generated by probe vibration or wristband pulses.

6. The system of claim 1 , wherein the computing device (102) implements end-to-end encryption using TLS or DTLS protocols for transmission of ultrasound and digital twin data.

7. The system of claim 1 , wherein the risk and guidance module computes safe injection zones based on patient-specific anatomy derived from the digital twin and real-time ultrasound, and assigns confidence scores to each recommended site.

8. A method for providing augmented reality-assisted ultrasound guidance for minimally invasive procedures, the method comprising the steps of:(a) acquiring real-time ultrasound images of a patient’s anatomy using a handheld ultrasound probe (101 );(b) transmitting the acquired images to a computing device (102) executing an image acquisition module for noise reduction and stabilization;(c) processing the images using an Al segmentation engine to identify anatomical structures including vessels and nerves and assigning confidence levels to each detected structure;(d) performing a calibration procedure using at least three fixed anatomical landmarks to align the ultrasound imaging plane with an augmented reality coordinate space;(e) generating a digital twin by combining 3D ultrasound data with surface scan data from an augmented reality display (103) to create a patient-specific three- dimensional anatomical model;(f) overlaying segmented structures and injection guidance indicators onto the patient view through the augmented reality display (103);(g) detecting a needle trajectory during an injection and issuing multimodal alerts (visual, auditory, or haptic) when the needle tip approaches a high-risk structure; and(h) updating the digital twin after the procedure with post-injection ultrasound data to record filler placement and anatomical changes.

9. The method of claim 8, wherein hybrid optical-IMU tracking is performed to determine probe orientation, and sensor fusion algorithms are applied to correct drift and maintain overlay alignment.

10. The method of claim 8, wherein the step of generating the digital twin includes acquiring 3D ultrasound data and surface scans to produce a composite model in STL or OBJ format within five minutes.

11. The method of claim 8, further comprising the step of simulating filler injection on the digital twin by dynamically updating the 3D model to visualize contour changes and potential vascular compression for different injection volumes.

12. The method of claim 8, wherein multimodal alerts comprise:(a) visual flashing overlays for high-risk proximity,(b) auditory warning tones, and(c) haptic pulses of varying intensity proportional to proximity risk.

13. The method of claim 8, wherein post-procedure ultrasound data are analyzed to update the digital twin and assess vascular perfusion using Doppler overlays rendered in the augmented reality display (103) J

Images