Infant airway management system and method based on navigation and high precision dynamic registration

Abstract

The application discloses a navigation and high-precision dynamic registration-based infant airway management system and method, wherein a physical constraint-based few-sample transfer learning network is utilized to construct a personalized airway deformation model and generate an optimal instrument matching scheme, a semantic weighted tight coupling visual inertial odometer is proposed, feature point confidence is dynamically adjusted in real time through real-time segmentation of airway anatomical semantics, and data is fused under a factor graph optimization framework to realize sub-millimeter level robust positioning. Based on the high-precision trajectory, a space-time attention fusion network is introduced into the system to capture long-range time sequence dependency between operation micro-motions and physiological parameters, calculate real-time dynamic risk scores, and compensate airway soft tissue deformation in real time through a non-rigid deformation field model. The application effectively solves the technical problems that existing navigation systems are prone to be lost in a micro dynamic environment and risk early warning is lagged, and significantly improves the safety and success rate of infant airway management surgery.

Description

Technical Field

[0001] This invention belongs to the field of medical device technology, specifically relating to an infant airway management system and method based on navigation and high-precision dynamic registration. Background Technology

[0002] Airway management in infants and young children, especially endotracheal intubation, is a high-risk procedure in pediatric anesthesia and emergency care. Its clinical challenges stem from the unique anatomy of the infant's airway: narrow, moist, and constantly moving. Key anatomical sites (such as the subglottic stenosis in three-month-old infants) have an effective diameter of only a few millimeters. Any minor operational error can lead to severe damage to the glottis or subglottic mucosa, causing acute edema, laryngospasm, and even long-term serious complications such as tracheal stenosis. Therefore, the core clinical goal of "preventing glottic and subglottic injury in infants and young children" requires navigation systems to achieve sub-millimeter level positioning and measurement accuracy, a stringent requirement for current technology.

[0003] To improve the safety and precision of surgeries, mixed reality (MR) navigation technology has been introduced into clinical practice, aiming to overlay a personalized 3D airway model constructed preoperatively onto the surgeon's field of vision in real time and accurately. However, existing general navigation technologies face insurmountable technical bottlenecks when applied to the specific clinical scenario of infant airways, making it difficult to achieve effective intraoperative real-time spatial registration and dynamic tracking.

[0004] 1. Lack of stable features: The airway wall is composed of smooth mucosal tissue, lacking the macroscopic geometric features such as corners and edges relied upon by traditional Simultaneous Localization and Mapping (SLAM) algorithms. Therefore, it is a macroscopically "feature-free" environment, making traditional SLAM algorithms prone to failure. This invention aims to solve this fundamental problem by creatively using mucosal microvascular texture and soft tissue folds as stable localization information sources.

[0005] 2. Dynamic and Non-rigid Environmental Changes: The child's spontaneous breathing, heartbeat, and contact with instruments all lead to complex non-rigid deformations of the airway soft tissues. Current airway navigation technologies primarily rely on rigid registration, neglecting the registration errors introduced by these drastic non-rigid deformations. Simultaneously, secretions on the mucosal surface and dramatic changes in illumination caused by light reflection further interfere with the stability of visual features.

[0006] 3. Real-time performance and accuracy bottlenecks: Existing general-purpose navigation solutions struggle to meet the stringent real-time requirements of surgery when processing complex airway data in infants and young children. The pose update frequency of the navigation system needs to reach the 100Hz level, and the visual feedback latency must be less than 100ms. Current technologies have significant shortcomings in real-time spatial registration and dynamic tracking.

[0007] 4. Fragmented Risk Assessment Mechanism: The existing navigation system and risk assessment system are fragmented, meaning "navigation only covers the route, while the monitor only covers the patient's life." The system lacks a mechanism to link the highly accurate calculated laryngoscope operation trajectory (such as vibration degree and hysteresis entropy) with the child's physiological parameters for causal inference and real-time risk warning.

[0008] In view of the aforementioned technical challenges, there is an urgent need in the field for a novel technical solution that can address the problems of low positioning accuracy, poor robustness, susceptibility to dynamic interference, and delayed risk warning in existing navigation technologies, thereby achieving high-precision real-time surgical navigation and ensuring the safety of clinical operations. This invention is proposed precisely to address these challenges. Summary of the Invention

[0009] To address the aforementioned technical problems, this invention proposes an infant airway management system based on navigation and high-precision dynamic registration, comprising:

[0010] The modeling and device recommendation module is used to generate personalized 3D airway baseline models and device recommendation parameters that meet physical prior conditions by using a transfer learning network that incorporates airway fluid dynamics constraints.

[0011] The pose calculation module is used to simultaneously acquire endoscopic images from a laryngoscope and data from an inertial measurement unit (IMU), perform customized enhancement processing on the endoscopic images from the laryngoscope, extract microscopic feature points of the airway mucosa, and calculate the real-time pose and motion trajectory of the laryngoscope tip.

[0012] The dynamic deformation field registration and compensation module is used to calculate and correct the deformation error in the 3D airway reference model caused by breathing or device compression, based on the topological correspondence between the microscopic feature points and the 3D airway reference model, by introducing the constraint of the non-rigid deformation field.

[0013] The risk assessment module is used to construct a multimodal spatiotemporal attention network. Taking the features of the motion trajectory and real-time physiological signals as input, it captures the causal delay features between operational behavior and physiological response through a cross-attention mechanism and outputs a real-time dynamic risk score.

[0014] The early warning module is used to map the corrected 3D airway baseline model onto the mixed reality terminal's field of view, making it overlap with the laryngoscope endoscopic image, and trigger a graded early warning based on the real-time dynamic risk score.

[0015] Specifically, in the transfer learning network of the modeling and device recommendation module, sparse image data of the child is mapped to a high-dimensional airway statistical manifold. This transfer learning network employs a conditional variational autoencoder (C-VAE) architecture. Its input receives feature vectors of the child, including age, weight, and Cormack-Lehane classification, along with the sparse image data. The output reconstructs a personalized... Airway baseline model.

[0016] Specifically, the device recommendation parameters in the modeling and device recommendation module are determined by calculating the device matching index. This device matching index is used to maximize the ratio of the outer diameter of the catheter to the narrowest point of the airway, and combined with the geometric factors of the curvature of the laryngoscope blade, to ensure that a predetermined safe gap is maintained between the catheter and the airway.

[0017] Specifically, the customized enhancement processing in the pose calculation module includes: performing red channel weighted grayscale processing on the acquired laryngoscope endoscopic image to highlight the mucosal microvascular texture, and then applying adaptive histogram equalization (CLAHE) to the grayscale image to enhance local contrast; the pose calculation module calculates the real-time pose and motion trajectory of the laryngoscope tip by performing tight coupling fusion through a factor graph optimization framework or an error state extended Kalman filter framework to calculate the real-time six-degree-of-freedom pose and motion trajectory of the laryngoscope tip.

[0018] Specifically, in the pose calculation module, the factor graph optimization framework constructs a joint cost function including prior factors, pre-integration factors, and semantically weighted visual reprojection factors; the joint cost function is dynamically modulated and visual reprojection errors are processed to achieve laryngoscope tip localization.

[0019] Specifically, the error state extended Kalman filter framework in the pose calculation module solves the positioning drift problem under rapid cannulation by fusing high-frequency inertial measurement unit (IMU) data with low-frequency visual data. To estimate and compensate for the positioning drift, the error state extended Kalman filter framework maintains an error state vector. Its linearized evolution equation is:

[0020] in This is the error state transition matrix, which explicitly describes the coupled influence of the zero bias and attitude errors of the inertial measurement unit (IMU) data on position and velocity. Represents the error state vector. middle,

[0021] Represents the noise Jacobian matrix. This represents the system noise vector.

[0022] Specifically, to achieve high-precision positioning, the pose calculation module is configured to: during visual observation updates, incorporate an external parameter transformation matrix to compensate for the physical offset between the optical center of the laryngoscope camera and the center of the inertial measurement unit (IMU), and combine it with zero-rate correction (ZUPT) logic to force zero-rate updates to suppress the integral drift of the IMU when the laryngoscope is in a static condition.

[0023] Specifically, the non-rigid deformation field in the dynamic deformation field registration and compensation module describes the displacement between the static 3D model point set and the intraoperative real-time point cloud. The non-rigid deformation field is solved by the coherent point drift (CPD) registration algorithm, thereby realizing dynamic compensation of airway soft tissue at the 3D modeling level.

[0024] Specifically, the multimodal spatiotemporal attention network (ST-AFN) in the risk assessment module captures causal delay features through the following mechanism:

[0025] A trajectory feature encoder is used to extract high-order kinematic features of the positioning trajectory output by the pose calculation module. The high-order kinematic features include at least jitter and hysteresis entropy.

[0026] The cross-attention module is used to define the physiological state as Query, and the operation trajectory vector includes a key vector Key and a value vector Value. It calculates the attention weight of operation action features on changes in physiological state in order to identify key operation segments that lead to increased risk.

[0027] This invention also discloses an infant airway management method based on navigation and high-precision dynamic registration, comprising:

[0028] Step 1: Using a transfer learning network that incorporates airway fluid dynamics constraints, a personalized 3D airway baseline model and recommended device parameters that satisfy the physical prior conditions are generated.

[0029] Step 2: Simultaneously acquire endoscopic images from the laryngoscope and data from the inertial measurement unit (IMU). Perform customized enhancement processing on the endoscopic images from the laryngoscope to extract microscopic feature points of the airway mucosa and calculate the real-time pose and motion trajectory of the laryngoscope tip.

[0030] Step 3: Based on the microscopic feature points and the topological correspondence with the 3D airway reference model, introduce the constraint of a non-rigid deformation field, calculate and correct the deformation error in the 3D airway reference model caused by breathing or device compression.

[0031] Step 4: Construct a multimodal spatiotemporal attention network. Using the features of the motion trajectory and real-time physiological signals as input, capture the causal delay features between operational behavior and physiological response through a cross-attention mechanism, and output a real-time dynamic risk score.

[0032] Step 5: Map the corrected 3D airway baseline model onto the mixed reality terminal's field of view, making it overlap with the laryngoscope endoscopic image, and trigger a graded warning based on the real-time dynamic risk score.

[0033] This invention proposes a semantically weighted tightly coupled visual-inertial odometry (SW-VIO), which significantly improves positioning performance in the confined, moist airway environment of infants and young children, where macroscopic features are lacking.

[0034] 1. Overcoming the limitations of "featureless" environments: The Micro-V-SLAM algorithm successfully identifies and extracts microscopic details such as microvascular textures and soft tissue folds on the airway mucosa surface as stable localization feature points, fundamentally solving the problem that traditional SLAM algorithms are prone to failure on macroscopic "featureless" mucosa.

[0035] 2. Intelligent Suppression of Dynamic Interference: The system introduces a semantic stability weighting mechanism within the Factor Graph Optimization (FGO) framework. A lightweight semantic segmentation network identifies anatomical semantics (blood vessels, glottis, secretions) in real time and dynamically adjusts the confidence weights of feature points based on the stability of their respective semantic meanings. Specifically, for dynamic interferences such as airway fluid flow and light reflection, the system approaches the weights of relevant feature points to zero, thus ignoring these outliers during optimization and significantly enhancing the system's robustness to dynamic environments.

[0036] 3. High-Frequency Tightly Coupled Fusion: The visual pose information from Micro-V-SLAM is tightly fused with data from an inertial measurement unit (IMU) embedded in the laryngoscope tip. The high-frequency characteristics of the IMU compensate for motion blur or feature loss that may occur in the visual algorithm during rapid movements (>10cm / s), achieving sub-millimeter accuracy with a spatial positioning error of less than 1mm. The system pose update frequency can reach up to 200Hz, meeting the stringent real-time requirements of surgical procedures.

[0037] This invention solves the registration error problem caused by the complex non-rigid deformation of the infant's airway due to breathing or instrument compression.

[0038] 1. Non-rigid deformation field calculation: The system constructs a non-rigid deformation field D(x) model based on the airway respiratory cycle and uses the coherent point drift (CPD) registration algorithm to calculate the deformation field in real time. This method performs non-rigid matching between the preoperative static 3D baseline model and the intraoperative real-time point cloud to calculate and correct the deformation error of the airway soft tissue.

[0039] 2. Real-time mapping and navigation: By compensating for deformation errors in real time, the system can accurately map the corrected 3D baseline model into the mixed reality (MR) field of view with sub-millimeter precision, realizing the dynamic and accurate overlay of the preoperative planning model and the real-time endoscopic image, i.e., the laryngoscope image, effectively guiding clinical operations.

[0040] This invention addresses the technical shortcomings of traditional systems' delayed risk warnings by deeply integrating operational trajectories and physiological states, thereby achieving proactive safety control.

[0041] 1. Causal Delay Feature Capture: A multimodal spatiotemporal attention fusion network (ST-AFN) was constructed. High-order kinematic features of the localization trajectory were extracted through a trajectory feature encoder, including jitter (rate of change of acceleration) and hysteresis entropy (degree of trajectory entanglement).

[0042] 2. Cross-Attention Risk Identification: A cross-attention module is employed, defining physiological state as Query and operational trajectory as Key and Value. This mechanism enables the system to calculate the attention weight of operational action features on changes in physiological state, thereby capturing the causal delay characteristics between "operational behavior and physiological response," identifying key operational segments that lead to increased risk, and achieving a breakthrough in overcoming the problem of traditional early warning lag.

[0043] 3. Active Physical Intervention: Based on the gradient of the real-time dynamic risk score (DRS), the system can trigger graded warnings on the MR terminal or perform variable impedance control through a force feedback device. As the risk increases, the stiffness and damping coefficients of the handle increase exponentially, forcibly slowing down high-risk operations (such as violent thrusting when approaching the subglottic stenosis) at the physical level, thus achieving an upgrade from passive navigation to active safety protection.

[0044] This invention introduces physical priors into preoperative planning, which improves the anatomical accuracy and reliability of the generated model.

[0045] 1. Physically Constrained Loss Function: The PINN-TL network introduces a physically based loss function term. This term constrains the generated 3D airway geometry to support a physically plausible airflow field, i.e., to satisfy the Navier-Stokes equations for airflow continuity at low Reynolds numbers. This ensures that the generated personalized airway model has topological plausibility and realistic dimensional reliability in terms of anatomy (e.g., the size of the subglottic stenosis).

[0046] 2. Intelligent instrument recommendation: Based on this high-precision personalized 3D model M, the system automatically calculates the instrument matching index to ensure that the outer diameter of the tube and the narrowest part of the airway maintain a safe gap, and achieves precise adaptation of the optimal laryngoscope blade curvature, length and endotracheal tube size. Attached Figure Description

[0047] Figure 1 A schematic diagram of an infant airway management system based on navigation and high-precision dynamic registration;

[0048] Figure 2 This is a schematic diagram of a navigation-based and high-precision dynamic registration-based method for infant airway management. Detailed Implementation

[0049] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] This invention proposes an infant airway management system based on navigation and high-precision dynamic registration, such as... Figure 1 As shown, it includes:

[0051] The modeling and device recommendation module utilizes a transfer learning network that incorporates airway fluid dynamics constraints to map sparse image data of children to a high-dimensional airway statistical manifold, generating a personalized 3D airway baseline model and device recommendation parameters that meet physical prior conditions.

[0052] A physically constrained few-shot transfer learning network (PINN-TL) is used to map sparse image data of children to a high-dimensional airway statistical manifold, generating personalized images with deformation properties. .

[0053] Network infrastructure: A conditional variational autoencoder (C-VAE) is built as the basic generative network to address the limitation of small sample size in pediatric data.

[0054] Input: Feature vector of the patient (Including age, weight, Cormack-Lehane classification, etc.) and sparse image data (such as X-rays or low-dose CT).

[0055] Output: Personalization Airway reference model It includes the precise dimensions of key components such as the subglottic stenosis.

[0056] To prevent the generation The model produces non-physical distortions or illogical geometric structures, leading to a loss of physical information in the system. The loss function constrains the generated geometry to support a physically reasonable airflow field (assuming that the airflow in the airway follows the laws of incompressible fluids), thereby ensuring that the dimensions of key components such as the subglottic stenosis are accurate and reliable.

[0057] Total loss function It is the loss of reconstruction KL divergence loss and physical information loss Weighted sum:

[0058]

[0059] in and This is a weighting balancing parameter used to adjust the contribution of each loss term.

[0060] Physical information loss It is based on the Navier-Stokes Equations, in the generated Internal sampling point set of the airway baseline model The computational fluid dynamics residuals satisfy the continuity equation and momentum equation of the airflow:

[0061]

[0062] in, Represents the reconstruction loss, which measures the matching accuracy between the generated model and the input image data;

[0063] This represents the KL divergence loss, used to regularize the latent space and ensure that the model can learn a smooth airway deformation manifold. Indicates that in the generation The point set sampled inside the airway baseline model is used to calculate the physical residual;

[0064] Indicates in The air velocity vector at the point, Represents the air pressure field. This represents the air density (constant). This represents the viscosity of air (a constant). This represents the divergence of the velocity field, corresponding to the residuals of the continuity equation. This represents the residual of the momentum equation.

[0065] Intelligent device recommendations are based on personalized recommendations generated by a sample transfer learning network (PINN-TL). Model The system automatically calculates the instrument matching index. This ensures that the recommended endotracheal tube size minimizes biases caused by reliance on experience or standard lookup tables in pediatric anesthesia and emergency care.

[0066] Device matching index Intended to ensure catheter outer diameter With the narrowest part of the airway Keep The above safety clearances:

[0067]

[0068] in, This indicates the recommended outer diameter of the endotracheal tube. Indicates by Model The effective diameter of the narrowest part of the child's airway, as measured, is usually in the subglottic segment. Indicates the laryngoscope blade The degree of matching between the curvature of the laryngoscope and the curvature of the child's airway is used to guide the selection of a laryngoscope. This represents a weighting factor used to balance the importance of catheter size matching and laryngoscope geometry matching. Optimize objectives and find ways to achieve them. Largest duct and laryngoscope blade combination, Indicates the type of endotracheal tube to be selected. This indicates the laryngoscope blade model to be selected.

[0069] The pose calculation module is used to simultaneously acquire endoscopic images from the laryngoscope and data from the inertial measurement unit (IMU). It performs customized image enhancement processing, extracts microscopic feature points of the airway mucosa, and calculates the sub-millimeter-level real-time six-degree-of-freedom pose and motion trajectory of the laryngoscope tip using a tightly coupled multi-sensor fusion algorithm. This module is used to fuse data from the high-frequency inertial measurement unit. With customization Visual data addresses the positioning drift issues in infants and young children with narrow airways and in highly dynamic environments, achieving a spatial positioning error of less than [value missing]. Sub-millimeter level positioning accuracy. The customized enhancement processing of the Micro-V-SLAM vision front-end system specifically includes: red channel weighted grayscale processing of the acquired endoscopic images to highlight the texture of mucosal microvessels, followed by applying adaptive histogram equalization (CLAHE) to the grayscale images to enhance local contrast.

[0070] The Micro-V-SLAM visual front-end system undergoes red channel weighted grayscale conversion and adaptive histogram equalization. From the enhanced images (such as those obtained through processing), microvascular textures and soft tissue folds on the airway mucosa surface are extracted as microscopic feature points. This is to optimize the laryngoscope's pose. The system constructs a nonlinear least squares optimization objective function to minimize the reprojection error. :

[0071]

[0072] The key lies in using the Cauchy robust kernel function. To handle the residuals of visual reprojection errors .

[0073] The Using the Cauchy kernel function, compared to the traditional Kernel function, for residuals The decay is faster, thus having a stronger suppressive effect on outliers during the optimization process. In the airway environment, outliers mainly originate from violent dynamic disturbances such as bubble bursting, rapid movement of secretions, or instantaneous tissue peristalsis. The dynamic constraint term is a penalty term introduced for non-rigid deformation of the airway, used to ensure that the pose calculation does not violate the laws of physical motion. The observed pixel coordinates of the feature point, i.e., the first pixel. The 2D coordinates of microscopic feature points, such as microvascular texture, actually detected on the plane of the current frame of the endoscopic image, i.e., the laryngoscope endoscopy image.

[0074] This represents the standard pinhole camera projection model, which describes the geometric mapping relationship between 3D spatial points projected onto a 2D image plane. The real-time pose matrix of the laryngoscope tip is represented, specifically including the transformation matrix from the world coordinate system to the camera coordinate system (including rotation R and translation t). Represents the coordinates of a feature point in 3D space, indicating the first... The three-dimensional positions of individual airway mucosal microscopic feature points in the world coordinate system. These points are typically obtained through triangulation or inverse depth estimation; Represents the square of the Mahalanobis distance. The photometric error term is represented by the grayscale information of the image (direct method) to assist in localization, ensuring that texture information is used to constrain the location in sparse feature point areas. , This represents the weighting balance coefficient.

[0075] Cauchyro bar kernel function The typical form is as follows:

[0076]

[0077] This kernel function effectively reduces the weight of incorrect matching in the optimization process, ensuring the accuracy and robustness of pose estimation.

[0078] The pose calculation module calculates the real-time pose and motion trajectory of the laryngoscope tip by: performing tight-coupled fusion through a factor graph optimization framework or an error state extended Kalman filter framework to calculate the sub-millimeter-level real-time six-DOF pose and motion trajectory of the laryngoscope tip. In the pose calculation module, the factor graph optimization framework constructs a joint cost function including prior factors, pre-integration factors, and semantically weighted visual reprojection factors; the joint cost function is dynamically modulated and visual reprojection errors are processed to achieve laryngoscope tip localization.

[0079] The pose calculation module's factor graph optimization framework includes a joint cost function, comprising a semantically weighted visual reprojection factor. The covariance matrix of this reprojection factor is modulated by a semantic stability function, which makes the weights of feature points related to airway secretions and specular reflections approach 0. A Cauchy robust kernel function ρ( This robust kernel function is used to handle visual reprojection errors. Compared with the Huber kernel, it has a stronger suppression effect on violent dynamic anomalies caused by airway bubble rupture or rapid movement of secretions, thus ensuring the accuracy of pose estimation.

[0080] The factor graph optimization framework is a nonlinear optimization backend in the microscopic visual inertial odometry (SW-VIO) method used to achieve tight-coupled fusion of multiple sensors and semantic enhancement. Semantically weighted microscopic visual inertial odometry (SW-VIO) is the core sensing module, designed to solve the problems of "unclear visibility and inaccurate positioning" in the airways of infants and young children by fusing high-frequency inertial data with semantically enhanced visual information, thereby achieving sub-millimeter-level robust positioning.

[0081] 1. Semantic segmentation front end and dynamic weighting of feature points

[0082] Input image After lightweight The network outputs a pixel-level semantic mask. Semantic category definitions include: For each feature point in the image Dynamic confidence weights are assigned based on their semantic category. For example, vascular texture is stable and has a high weight; while fluid and reflection have weights close to 0.

[0083] 2. Factor Graph Optimization Backend and Joint Cost Function

[0084] The system uses a sliding window factor graph to analyze state variables. Perform joint optimization. The optimization objective is to minimize the sum of squared residuals of all factors, expressed as follows:

[0085]

[0086] in As a priori factor, For the IMU pre-integration factor, the third term The semantically weighted visual reprojection factor. That is, the residual vector of visual reprojection error. ( ) represents the set of state variables that minimizes the total cost function (i.e., the sum of squared residuals of all factors). It is a set of state variables to be optimized within a sliding window, used to describe the movement and environment of the laryngoscope tip. Represents the IMU pre-integration residual vector. For IMU observations, denoted as angular velocity and acceleration measured by the IMU in the k-th frame. Denotes the prior residual vector. Denotes the prior Jacobian matrix. This represents the Cauchy robust kernel function. The joint cost function consists of three main parts, corresponding to prior knowledge, inertial observations, and semantically enhanced visual observations. Semantic weighting is achieved by dynamically modulating the visual reprojection factor. covariance matrix accomplish:

[0087]

[0088] in, It is a semantic stability function. Through this modulation, the system makes the weights of feature points related to airway secretion flow and specular reflection approach a certain value. Thus The system tightly locks onto the microvascular texture on the vessel wall, thereby suppressing dynamic interference.

[0089] In the joint cost function middle, This represents the Cauchy robust kernel function, used to handle visual reprojection errors. This robust kernel function is compared to... The kernel provides stronger suppression of outliers. Robust kernel function. The specific form is as follows:

[0090]

[0091] This kernel function is used to deal with drastic dynamic anomalies in the airway caused by bubble rupture or rapid movement of secretions. It can effectively reduce the weight of mismatches in the optimization process, thereby ensuring the accuracy of pose estimation.

[0092] This system preferably uses an error state extended Kalman filter (ESB). )Frame for visual and inertial measurement units Tightly coupled data fusion, with a fusion frequency of no less than The system utilizes an inertial measurement unit. High-frequency data (up to This compensates for motion blur or feature loss in visual algorithms during rapid movement, ensuring that spatial positioning errors are controlled within a preset range. Within a range of [values]. This involves maintaining a state vector describing the motion state of the laryngoscope tip. To simplify nonlinear processing, the system will use the real state. Decomposed into nominal states and error state .

[0093] Nominal state vector (True value estimation) includes:

[0094]

[0095] in For position and velocity; For attitude quaternions; These are the zero bias of the accelerometer and the zero bias of the gyroscope, respectively.

[0096] Error state vector The error term that needs to be estimated online is defined as follows: Dimensional vector:

[0097]

[0098] in The angular error is expressed in Lie algebra form. For positional error, For speed error, To reduce the zero bias error of the accelerometer, This refers to the zero bias error of the gyroscope.

[0099] The pose calculation module's error state extended Kalman filter framework solves the positioning drift problem during rapid cannulation by fusing high-frequency inertial measurement unit (IMU) data with low-frequency visual data. The prediction model updates the nominal state using Newtonian kinematics. Specifically, to estimate and compensate for the positioning drift, the error state extended Kalman filter framework maintains an error state vector. Its linearized evolution equation is:

[0100]

[0101] in This is the error state transition matrix, which explicitly describes the coupled influence of the zero bias and attitude errors of the inertial measurement unit (IMU) data on position and velocity. Represents the error state vector.

[0102] Represents the noise Jacobian matrix. This represents the system noise vector.

[0103] in, For noise vectors, and The error state transition matrix explicitly describes the inertial measurement unit. The coupling effect of zero bias and attitude errors on position and velocity is addressed, thus resolving the state coupling problem in multi-sensor fusion. The linearization evolution equation and transition matrix of the error state are presented. The specific form is as follows:

[0104]

[0105] in, This represents the antisymmetric matrix of vectors, used to calculate the cross product. This matrix ensures [property / efficacy] by explicitly modeling the coupling of error terms. Convergence and accuracy under high-frequency, high-dynamic environments. 0 3×3 and I 3×3 Let these represent the 3×3 zero matrix and identity matrix, respectively. R ( q ^) is a nominal attitude quaternion q ^The rotation matrix derived from the transformation. and These are the acceleration and angular velocity measured by the inertial measurement unit (IMU), respectively. and These are the estimated values ​​for the accelerometer zero bias and the gyroscope zero bias, respectively. ] × The skew-symmetric matrix represents the vector used to calculate the cross product. This matrix explicitly linearizes the propagation relationship between error states (such as attitude errors) and velocity and position, and is key to high-precision state prediction and covariance matrix updates within the ES-EKF framework. Fx Jacobian matrix. To further ensure sub-millimeter accuracy, the system also includes the following key compensation measures:

[0106] Extrinsic parameter compensation: incorporating the extrinsic parameter transformation matrix The optical center of the laryngoscope camera and the inertial measurement unit The physical offset between centers (i.e., the "lever effect") is compensated for.

[0107] Zero speed correction ( When the laryngoscope is detected to meet static conditions, the system forces a zero-rate update observation. This strong constraint suppresses the inertial measurement unit. Position drift is minimized to ensure sub-millimeter accuracy during extended operation.

[0108] The dynamic deformation field registration and compensation module, based on the microscopic feature points of the airway mucosa and the topological correspondence with the 3D airway reference model, introduces the constraint of a non-rigid deformation field to calculate and correct the deformation error of the airway soft tissue caused by breathing or instrument compression. This step is used to solve the technical bottleneck of existing navigation systems that mainly rely on rigid registration and ignore the severe non-rigid deformation of the infant's airway caused by breathing and instrument compression, so as to ensure a high degree of consistency between the preoperative planning model and the real-time airway morphology during surgery.

[0109] The system treats airway deformation as a non-rigid displacement field in space. Preoperative access is defined as... Obtained static The baseline model point set is During the procedure, the pose calculation module ( Real-time construction of sparse point clouds based on microscopic feature points .

[0110] The non-rigid deformation field Describes static Benchmark model point set Sparse point cloud to microscopic feature points The displacement relationship between them is modeled as follows:

[0111]

[0112] in, It is the non-rigid deformation field to be solved, describing the displacement of each point in space; To observe noise or registration residuals.

[0113] The establishment of the correspondence between data sources and topology includes the following steps: sparse point cloud of microscopic feature points. The construction relies on the extraction and tracking of microscopic feature points of the airway mucosa by the pose calculation module, such as microvascular textures and soft tissue folds. These feature points are triangulated or their three-dimensional coordinates in the world coordinate system are obtained through visual-inertial fusion or inverse depth estimation.

[0114] exist During the modeling phase, it has become static. Benchmark model point set It assigns anatomical semantic annotations and topological information. The system uses local geometric descriptors (e.g., or Sparse point cloud matching microscopic feature points Microscopic feature points and static Baseline model point set Establish topological correspondences between point sets based on the corresponding anatomical locations.

[0115] In the process of deformation field calculation and dynamic compensation, a non-rigid registration algorithm is used for calculation. In order to find Transform to The optimal non-rigid transformation. This embodiment preferably employs the Coherent Point Drift (CPD) algorithm, or a similar algorithm based on a Gaussian mixture model (Gaussian mixture model). The registration algorithm is used for solving.

[0116] The coherence point drift The algorithm transforms the registration problem into maximization. The problem of likelihood estimation is to minimize the deformation field. regularization term and data fitting terms Joint energy function:

[0117]

[0118] in, These are regularization weights used to balance the accuracy of point set matching and the smoothness of the deformation field.

[0119] Dynamic compensation and The superposition process is achieved through the following steps: once the deformation field... Once solved, it will be applied in real time. After being corrected Model :

[0120]

[0121] Finally, the system will correct this. The model, combined with the precise pose of the laryngoscope output by the pose calculation module, is mapped to mixed reality with sub-millimeter accuracy. Within the field of view, it achieves precise overlay of virtual and real elements, thereby providing a dynamic and real-time navigation view.

[0122] The risk assessment module is used to construct a multimodal spatiotemporal attention network. Taking the characteristics of the motion trajectory and real-time physiological signals as input, it captures the causal delay characteristics between the operation behavior and physiological response through a cross-attention mechanism and outputs a real-time dynamic risk score (DRS).

[0123] This module addresses the technical challenge of delayed clinical risk warnings (i.e., when physiological signs deteriorate, operational errors have often already occurred). It utilizes a spatiotemporal attention fusion network-based approach. The model aims to address the causal delay between procedural actions and physiological responses in clinical risk warning systems. The system can predict procedural trajectories. For subsequent physiological state The potential impact, and output real-time dynamic risk scores in advance ( ).

[0124] Multimodal feature engineering and input space, specifically including: system construction of multimodal input sequences. The operation trajectory High precision obtained from the pose calculation module Exporting pose sequences.

[0125] Trajectory Feature Encoder ( The localization trajectory (position) output by the pose calculation module is extracted using a one-dimensional convolutional layer. Higher-order kinematic characteristics are used to quantify the operator's hand stability and hesitation; these higher-order kinematic characteristics include:

[0126] jitter : Defined as the rate of change of acceleration at the tip of the laryngoscope, reflecting the stability of the operator's hand.

[0127]

[0128] in This is the acceleration of the laryngoscope tip. is the position vector of the laryngoscope tip.

[0129] Hesitation entropy: A nonlinear index that reflects operator hesitation and inefficient repetitive operations by calculating the degree of entanglement of a trajectory within a unit volume or the rate of change of local curvature.

[0130] Other features include local curvature and velocity. ).

[0131] Physiological timing encoder ( ) Utilizing Long Short-Term Memory Networks ( Recurrent neural networks (RNNs) are used to extract vital signs such as oxygen saturation (SpO2), heart rate (HR), and time-dependent features of end-tidal carbon dioxide partial pressure (ETCO2).

[0132] Cross-attention is a spatiotemporal attention mechanism designed to capture causal delays between behavioral actions and physiological responses, such as those resulting from strenuous activity. What happened But physiological deterioration What happened At any time, the system adopts Cross-attention module of the architecture .

[0133] Query, Key, Value definitions: , ,

[0134] Query( ): Defined as a physiological state The encoded representation is derived from the child's real-time physiological signals (oxygen saturation SpO2, heart rate HR, end-tidal carbon dioxide partial pressure ETCO2). The parameters derived from neural network training are used to map input features to the Q, K, V space.

[0135] Key( ) and Value( ): Defined as the operation trajectory The encoded representation.

[0136] Key( ) is the key vector, which originates from the higher-order kinematic features (such as jerkiness and hesitation entropy) output by the pose calculation module; Value( ) is a value vector, which carries specific information about the operation behavior.

[0137] Attention computation and causal inference, where the cross-attention module calculates... and The correlation is used to generate an attention weight matrix. The matrix Intuitively representing the first The operation action at time 1 The contribution of physiological risk at any given moment is used to calculate the attentional weight of operational behavior to changes in physiological state, thereby identifying key operational segments that lead to increased risk.

[0138]

[0139] in, yes The dimension of a vector. Specifically, this means identifying key historical operational segments that lead to increased current physiological risk by calculating the attention weight of physiological states to operational trajectories; the system constructs multimodal input sequences. The mapping is represented by Q, K, and V vectors, which clearly distinguishes the roles of operational behaviors and physiological states.

[0140] Dynamic Risk Score (DRS) prediction is achieved by inputting the feature vector fused through cross-attention to the output layer of the network, connecting... Activation function, outputs real-time dynamic risk score ( ) :

[0141] This dynamic risk score prediction Can be presented directly The terminal triggers tiered warnings based on the scoring gradient.

[0142] In addition, networks utilize multi-task learning ( The system outputs an auxiliary predictive metric: "Estimated Remaining Safe Operating Time" (Time-to-Desaturation). This metric quantifies the predicted vital signs (such as oxygen saturation) of the child under the current operating trend. When is it expected that the price will fall below a preset safety threshold (e.g.) This provides doctors with more timely decision support.

[0143] The early warning module is used to map the corrected 3D reference model to the mixed reality (MR) field of view with sub-millimeter accuracy, so that it overlaps with the real-time endoscope image, i.e. the laryngoscope endoscope image, and trigger a graded early warning on the mixed reality MR terminal according to the DRS scoring gradient.

[0144] This embodiment describes how to dynamically register... The model (virtual information) is fused with real-time endoscopic images (real information), and based on... The output risk score in mixed reality ( The display terminal provides real-time, tiered warnings and navigation guidance.

[0145] Geometric registration and precision assurance through virtual-real overlay During projection, the system utilizes the deformation-corrected output from the dynamic deformation field registration and compensation module. benchmark model The high-precision six-DOF pose calculated by the pose calculation module ,Will Each point Projected to Display terminal pixel coordinates superior.

[0146] The projection process follows a standard pinhole camera model. :

[0147]

[0148] in, This is the inverse pose transformation matrix from the laryngoscope tip to the world coordinate system. Because the pose calculation module guarantees a positioning accuracy error of less than [value missing]. Furthermore, the dynamic deformation field registration and compensation module compensates for tissue deformation, thus achieving sub-millimeter-level precise overlap between the navigation line and the glottis position, significantly improving... The accuracy of navigation maps.

[0149] To meet the stringent requirements of hand-eye coordination during surgical procedures, the overall visual feedback delay of the virtual-real overlay is... Must be controlled within Within this range. This relies on high-frequency multi-sensor fusion (fusion frequency can reach up to...). The low-latency, high-update-rate pose data provided.

[0150] The system visualizes the real-time dynamic risk score (DRS) output by the risk assessment module. Integrate into Intelligent decision support is implemented in the navigation view.

[0151] A tiered early warning mechanism is adopted, based on The system triggers tiered alerts based on real-time data to remind operators to adjust their techniques.

[0152] low-risk areas ): Provides smooth green or blue navigation lines to indicate the best route.

[0153] medium-risk areas ): The system is in A yellow warning bar appears at the edge of the field of vision, accompanied by a voice prompt indicating increased risk. The navigation line color changes to yellow or orange.

[0154] High-risk areas ( ):exist A red warning pops up in the center of the field of vision, and high-risk anatomical structures (such as subglottic stenosis) are highlighted by color changes or flashing, forcing the operator to stop or withdraw the operation immediately.

[0155] Risk indicator visualization is in On the terminal, in addition to displaying In addition, the system also displays the following auxiliary forecasting indicators simultaneously:

[0156] Operation trajectory deviation: The current operation trajectory is indicated by a color gradient. The deviation distance of the model's optimal path planning.

[0157] Estimated remaining safe operating time (Time-to-Desaturation): This is The auxiliary output is in the form of a countdown timer (e.g., "There is also..."). Within seconds of safe operation time, blood oxygen levels will drop below [a certain value]. This visually indicates the physiological limits.

[0158] The system utilizes the modeling and instrument recommendation modules. Anatomical semantic annotation of the benchmark model, in Key vulnerable structures (such as the vocal cords and subglottic stenosis) are geometrically highlighted in the field of view. As the laryngoscope tip approaches these structures, the virtual projection outline dynamically changes color and transparency, providing visual protection for the core targets of preventing damage to the glottis and subglottic region in infants and young children.

[0159] This invention proposes a method for infant airway management based on navigation and high-precision dynamic registration, such as... Figure 2 As shown, it includes the following steps:

[0160] Step 1: Using a transfer learning network that incorporates airway hydrodynamic constraints, the sparse image data of the child is mapped to a high-dimensional airway statistical manifold to generate a personalized 3D airway baseline model and recommended device parameters that meet the physical prior conditions.

[0161] A physically constrained few-shot transfer learning network (PINN-TL) is used to map sparse image data of children to a high-dimensional airway statistical manifold, generating personalized data with deformation properties. .

[0162] Network infrastructure: A conditional variational autoencoder (C-VAE) is built as the basic generative network to address the limitation of small sample size in pediatric data.

[0163] Input: Feature vector of the patient (Including age, weight, Cormack-Lehane classification, etc.) and sparse image data (such as X-rays or low-dose CT).

[0164] Output: Personalized airway It includes the precise dimensions of key components such as the subglottic stenosis.

[0165] To prevent the generation The model produces non-physical distortions or illogical geometric structures, leading to a loss of physical information in the system. The loss function constrains the generated geometry to support a physically reasonable airflow field (assuming that the airflow in the airway follows the laws of incompressible fluids), thereby ensuring that the dimensions of key components such as the subglottic stenosis are accurate and reliable.

[0166] Total loss function It is the loss of reconstruction KL divergence loss and physical information loss Weighted sum:

[0167]

[0168] in and This is a weighting balancing parameter used to adjust the contribution of each loss term.

[0169] Physical information loss It is based on the Navier-Stokes Equations, in the generated Internal sampling point set The computational fluid dynamics residuals satisfy the continuity equation and momentum equation of the airflow:

[0170]

[0171] in, Represents the reconstruction loss, which measures the matching accuracy between the generated model and the input image data;

[0172] This represents the KL divergence loss, used to regularize the latent space and ensure that the model can learn a smooth airway deformation manifold. Indicates that in the generation The internally sampled point set is used to calculate the physical residual;

[0173] Indicates in The air velocity vector at the point, Represents the air pressure field. This represents the air density (constant). This represents the viscosity of air (a constant). This represents the divergence of the velocity field, corresponding to the residuals of the continuity equation. This represents the residual of the momentum equation.

[0174] Intelligent device recommendations are based on personalized recommendations generated by a sample transfer learning network (PINN-TL). Model The system automatically calculates the instrument matching index. This ensures that the recommended endotracheal tube size minimizes biases caused by reliance on experience or standard lookup tables in pediatric anesthesia and emergency care.

[0175] Device matching index Intended to ensure catheter outer diameter With the narrowest part of the airway Keep The above safety clearances:

[0176]

[0177] in, This indicates the recommended outer diameter of the endotracheal tube. Indicates by Model The effective diameter of the narrowest part of the child's airway (usually in the subglottic segment). Indicates the laryngoscope blade The degree of matching between the curvature of the laryngoscope and the curvature of the child's airway is used to guide the selection of a laryngoscope. This represents a weighting factor used to balance the importance of catheter size matching and laryngoscope geometry matching. Optimize objectives and find ways to achieve them. Largest duct and laryngoscope blade combination; Indicates the type of endotracheal tube to be selected. This indicates the laryngoscope blade model to be selected.

[0178] Step 2: Simultaneously acquire endoscopic images from the laryngoscope and data from the inertial measurement unit (IMU). Perform customized enhancement processing on the images, extract microscopic feature points of the airway mucosa, and calculate the sub-millimeter-level real-time six-degree-of-freedom pose and motion trajectory of the laryngoscope tip using a tightly coupled multi-sensor fusion algorithm. This step is used to fuse high-frequency inertial measurement unit data. With customization Visual data addresses the positioning drift issues in infants and young children with narrow airways and in highly dynamic environments, achieving a spatial positioning error of less than [value missing]. Sub-millimeter level positioning accuracy. The customized enhancement processing of the Micro-V-SLAM vision front-end system specifically includes: red channel weighted grayscale processing of the acquired endoscopic images to highlight the texture of mucosal microvessels, followed by applying adaptive histogram equalization (CLAHE) to the grayscale images to enhance local contrast.

[0179] The Micro-V-SLAM visual front-end system undergoes red channel weighted grayscale conversion and adaptive histogram equalization. From the enhanced images (such as those obtained through processing), microvascular textures and soft tissue folds on the airway mucosa surface are extracted as microscopic feature points. This is to optimize the laryngoscope's pose. The system constructs a nonlinear least squares optimization objective function to minimize the reprojection error. :

[0180]

[0181] The key lies in using the Cauchy robust kernel function. To handle the residuals of visual reprojection errors .

[0182] The Using the Cauchy kernel function, compared to the traditional Kernel function, for residuals The decay is faster, thus having a stronger suppressive effect on outliers during the optimization process. In the airway environment, outliers mainly originate from violent dynamic disturbances such as bubble bursting, rapid movement of secretions, or instantaneous tissue peristalsis. The dynamic constraint term is a penalty term introduced for the non-rigid deformation of the airway, used to ensure that the pose calculation does not violate the laws of physical motion. The observed pixel coordinates of the feature point, i.e., the first pixel. The actual 2D coordinates of microscopic feature points, such as microvascular texture, detected on the plane of the current frame of the endoscopic image.

[0183] This represents the standard pinhole camera projection model, which describes the geometric mapping relationship between 3D spatial points projected onto a 2D image plane. The real-time pose matrix of the laryngoscope tip is represented, specifically including the transformation matrix from the world coordinate system to the camera coordinate system (including rotation R and translation t). Represents the coordinates of a feature point in 3D space, indicating the first... The three-dimensional positions of individual airway mucosal microscopic feature points in the world coordinate system. These points are typically obtained through triangulation or inverse depth estimation; Represents the square of the Mahalanobis distance. The photometric error term is represented by the grayscale information of the image (direct method) to assist in localization, ensuring that texture information is used to constrain the location in sparse feature point areas. , This represents the weighting balance coefficient.

[0184] Cauchyro bar kernel function The typical form is as follows:

[0185]

[0186] This kernel function effectively reduces the weight of incorrect matching in the optimization process, ensuring the accuracy and robustness of pose estimation.

[0187] In this step, calculating the real-time pose and trajectory of the laryngoscope tip includes: using a factor graph optimization framework or an error-state extended Kalman filter framework for tight coupling fusion to calculate the sub-millimeter-level real-time six-DOF pose and trajectory of the laryngoscope tip. In this step, the factor graph optimization framework constructs a joint cost function including prior factors, pre-integration factors, and semantically weighted visual reprojection factors; the joint cost function is dynamically modulated and visual reprojection errors are processed to achieve laryngoscope tip localization.

[0188] The factor graph optimization framework in this step includes a joint cost function, comprising a semantically weighted visual reprojection factor. The covariance matrix of the reprojection factor is modulated by a semantic stability function, which makes the weights of feature points related to airway secretions and specular reflections approach 0. A Cauchy robust kernel function ρ( This robust kernel function is used to handle visual reprojection errors. Compared with the Huber kernel, it has a stronger suppression effect on violent dynamic anomalies caused by airway bubble rupture or rapid movement of secretions, thus ensuring the accuracy of pose estimation.

[0189] The factor graph optimization framework is a nonlinear optimization backend in the microscopic visual inertial odometry (SW-VIO) method used to achieve tight-coupled fusion of multiple sensors and semantic enhancement. Semantically weighted microscopic visual inertial odometry (SW-VIO) is the core sensing module, designed to solve the problems of "unclear visibility and inaccurate positioning" in the airways of infants and young children by fusing high-frequency inertial data with semantically enhanced visual information, thereby achieving sub-millimeter-level robust positioning.

[0190] 1. Semantic segmentation front end and dynamic weighting of feature points

[0191] Input image After lightweight The network outputs a pixel-level semantic mask. Semantic category definitions include: For each feature point in the image Dynamic confidence weights are assigned based on their semantic category. For example, vascular texture is stable and has a high weight; while fluid and reflection have weights close to 0.

[0192] 2. Factor Graph Optimization Backend and Joint Cost Function

[0193] The system uses a sliding window factor graph to analyze state variables. Perform joint optimization. The optimization objective is to minimize the sum of squared residuals of all factors, expressed as follows:

[0194]

[0195] in As a priori factor, For the IMU pre-integration factor, the third term The semantically weighted visual reprojection factor. That is, the residual vector of visual reprojection error. ( ) represents the set of state variables that minimizes the total cost function (i.e., the sum of squared residuals of all factors). It is a set of state variables to be optimized within a sliding window, used to describe the movement and environment of the laryngoscope tip. Represents the IMU pre-integration residual vector. For IMU observations, denoted as angular velocity and acceleration measured by the IMU in the k-th frame. Denotes the prior residual vector. Denotes the prior Jacobian matrix. This represents the Cauchy robust kernel function. The joint cost function consists of three main parts, corresponding to prior knowledge, inertial observations, and semantically enhanced visual observations. Semantic weighting is achieved by dynamically modulating the visual reprojection factor. covariance matrix accomplish:

[0196]

[0197] in, It is a semantic stability function. Through this modulation, the system makes the weights of feature points related to airway secretion flow and specular reflection approach a certain value. Thus The system tightly locks onto the microvascular texture on the vessel wall, thereby suppressing dynamic interference.

[0198] In the joint cost function middle, This represents the Cauchy robust kernel function, used to handle visual reprojection errors. This robust kernel function is compared to... The kernel provides stronger suppression of outliers. Robust kernel function. The specific form is as follows:

[0199]

[0200] This kernel function is used to deal with drastic dynamic anomalies in the airway caused by bubble rupture or rapid movement of secretions. It can effectively reduce the weight of mismatches in the optimization process, thereby ensuring the accuracy of pose estimation.

[0201] This system preferably uses an error state extended Kalman filter (ESB). ) Framework for visual and Tightly coupled data fusion, with a fusion frequency of no less than The system utilizes an inertial measurement unit. High-frequency data (up to This compensates for motion blur or feature loss in visual algorithms during rapid movement, ensuring that spatial positioning errors are controlled within a preset range. Within a range of [values]. This involves maintaining a state vector describing the motion state of the laryngoscope tip. To simplify nonlinear processing, the system will use the real state. Decomposed into nominal states and error state .

[0202] Nominal state vector (True value estimation) includes:

[0203]

[0204] in For position and velocity; For attitude quaternions; These are the zero bias of the accelerometer and the zero bias of the gyroscope, respectively.

[0205] Error state vector The error term that needs to be estimated online is defined as follows: Dimensional vector:

[0206]

[0207] in The angular error is expressed in Lie algebra form. For positional error, For speed error, To reduce the zero bias error of the accelerometer, This refers to the zero bias error of the gyroscope.

[0208] The aforementioned error state extended Kalman filter framework addresses the positioning drift problem during rapid cannulation by fusing high-frequency inertial measurement unit (IMU) data with low-frequency visual data. (Inertial Measurement Unit) The prediction model updates the nominal state using Newtonian kinematics. Specifically, to estimate and compensate for the positioning drift, the error state extended Kalman filter framework maintains an error state vector. Its linearized evolution equation is:

[0209]

[0210] in This is the error state transition matrix, which explicitly describes the coupled influence of the zero bias and attitude errors of the inertial measurement unit (IMU) data on position and velocity. Represents the error state vector.

[0211] Represents the noise Jacobian matrix. This represents the system noise vector. The error state transition matrix explicitly describes the inertial measurement unit. The coupling effect of zero bias and attitude error on position and velocity is solved, thus addressing the state coupling problem in multi-sensor fusion. The specific form is as follows:

[0212]

[0213] in, This represents the antisymmetric matrix of vectors, used to calculate the cross product. This matrix ensures [property / efficacy] by explicitly modeling the coupling of error terms. Convergence and accuracy under high-frequency, high-dynamic environments. 0 3×3 and I 3×3 Let these represent the 3×3 zero matrix and identity matrix, respectively. R ( q ^) is a nominal attitude quaternion q ^The rotation matrix derived from the transformation. and These are the acceleration and angular velocity measured by the IMU, respectively. and These are the estimated values ​​for the accelerometer zero bias and the gyroscope zero bias, respectively. ] × The skew-symmetric matrix represents the vector used to calculate the cross product. This matrix explicitly linearizes the propagation relationship between error states (such as attitude errors) and velocity and position, and is key to high-precision state prediction and covariance matrix updates within the ES-EKF framework. Fx Jacobian matrix. To further ensure sub-millimeter accuracy, the system also includes the following key compensation measures:

[0214] Extrinsic parameter compensation: incorporating the extrinsic parameter transformation matrix The optical center of the laryngoscope camera and the inertial measurement unit The physical offset between centers (i.e., the "lever effect") is compensated for.

[0215] Zero speed correction ( When the laryngoscope is detected to meet static conditions, the system forces a zero-rate update observation. This strong constraint suppresses the inertial measurement unit. Position drift is minimized to ensure sub-millimeter accuracy during extended operation.

[0216] Step 3: Based on the microscopic feature points of the airway mucosa and the topological correspondence with the 3D airway reference model, a non-rigid deformation field constraint is introduced to calculate and correct the deformation error of the airway soft tissue caused by breathing or instrument compression. This step is used to solve the technical bottleneck of existing navigation systems that mainly rely on rigid registration and ignore the severe non-rigid deformation of the infant's airway caused by breathing and instrument compression, so as to ensure a high degree of consistency between the preoperative planning model and the real-time airway morphology during surgery.

[0217] The system treats airway deformation as a non-rigid displacement field in space. Preoperative access is defined as... Obtained static The baseline model point set is During the operation, step 2 ( Real-time construction of sparse point clouds based on microscopic feature points .

[0218] The non-rigid deformation field Describes static Baseline model point set Sparse point cloud to microscopic feature points The displacement relationship between them is modeled as follows:

[0219]

[0220] in, It is the non-rigid deformation field to be solved, describing the displacement of each point in space; To observe noise or registration residuals.

[0221] The establishment of the correspondence between data sources and topology includes the following steps: sparse point cloud of microscopic feature points. The construction relies on the microscopic features of the airway mucosa extracted and tracked in step 2, such as microvascular textures and soft tissue folds. These features are then triangulated or their three-dimensional coordinates in the world coordinate system are obtained through visual-inertial fusion or inverse depth estimation.

[0222] exist During the modeling phase, it has become static. Baseline model point set It assigns anatomical semantic annotations and topological information. The system uses local geometric descriptors (e.g., or Sparse point cloud matching microscopic feature points Microscopic feature points and static Benchmark model point set Establish topological correspondences between point sets based on the corresponding anatomical locations.

[0223] In the process of deformation field calculation and dynamic compensation, a non-rigid registration algorithm is used for calculation. In order to find Transform to The optimal non-rigid transformation. This embodiment preferably employs the Coherent Point Drift (CPD) algorithm, or a similar algorithm based on a Gaussian mixture model (Gaussian mixture model). The registration algorithm is used for solving.

[0224] The coherence point drift The algorithm transforms the registration problem into maximization. The problem of likelihood estimation is to minimize the deformation field. regularization term and data fitting terms Joint energy function:

[0225]

[0226] in, These are regularization weights used to balance the accuracy of point set matching and the smoothness of the deformation field.

[0227] Dynamic compensation and The superposition process is achieved through the following steps: once the deformation field... Once solved, it will be applied in real time. After being corrected Model :

[0228]

[0229] Finally, the system will correct this. The model, combined with the precise pose of the laryngoscope output in step 2, is mapped to mixed reality with sub-millimeter precision. Within the field of view, it achieves precise overlay of virtual and real elements, thereby providing a dynamic and real-time navigation view.

[0230] Step 4: Construct a multimodal spatiotemporal attention network. Using the features of the motion trajectory and real-time physiological signals as input, capture the causal delay features between operational behavior and physiological response through a cross-attention mechanism, and output a real-time dynamic risk score (DRS).

[0231] This step addresses the technical challenge of delayed clinical risk warnings (i.e., when physiological signs deteriorate, the operational error has often already occurred). It utilizes a spatiotemporal attention fusion network-based approach. The model aims to address the causal delay between procedural actions and physiological responses in clinical risk warning systems. The system can predict procedural trajectories. For subsequent physiological state The potential impact, and output real-time dynamic risk scores in advance ( ).

[0232] Multimodal feature engineering and input space, specifically including: system construction of multimodal input sequences. The operation trajectory The high precision obtained in step 2 Exporting pose sequences.

[0233] Trajectory Feature Encoder ( The localization trajectory (position) output from step 2 is extracted using a one-dimensional convolutional layer. The higher-order kinematic characteristics are used to quantify the operator's hand stability and hesitation; these higher-order kinematic characteristics include:

[0234] jitter : Defined as the rate of change of acceleration at the tip of the laryngoscope, reflecting the stability of the operator's hand.

[0235]

[0236] in This is the acceleration of the laryngoscope tip. is the position vector of the laryngoscope tip.

[0237] Hesitation entropy: A nonlinear index that reflects operator hesitation and inefficient repetitive operations by calculating the degree of entanglement of a trajectory within a unit volume or the rate of change of local curvature.

[0238] Other features include local curvature and velocity. ).

[0239] Physiological timing encoder ( ) Utilizing Long Short-Term Memory Networks ( Recurrent neural networks (RNNs) are used to extract vital signs such as oxygen saturation (SpO2), heart rate (HR), and time-dependent features of end-tidal carbon dioxide partial pressure (ETCO2).

[0240] Cross-attention is a spatiotemporal attention mechanism designed to capture causal delays between behavioral actions and physiological responses, such as those resulting from strenuous activity. What happened But physiological deterioration What happened At any time, the system adopts Cross-attention module of the architecture .

[0241] Query, Key, Value definitions: , ,

[0242] Query( ): Defined as a physiological state The encoded representation is derived from the child's real-time physiological signals, including oxygen saturation (SpO2), heart rate (HR), and end-tidal carbon dioxide partial pressure (ETCO2). The parameters derived from neural network training are used to map input features to the Q, K, V space.

[0243] Key( ) and Value( ): Defined as the operation trajectory The encoded representation.

[0244] Key( The source of ) is the higher-order kinematic features output from step 2 (such as jerkiness and hesitation entropy); Value( ) carries specific information about the operation.

[0245] Attention computation and causal inference, where the cross-attention module calculates... and The correlation is used to generate an attention weight matrix. The matrix Intuitively representing the first The operation action at time 1 The contribution of physiological risk at any given moment is used to calculate the attentional weight of operational behavior to changes in physiological state, thereby identifying key operational segments that lead to increased risk.

[0246]

[0247] in, yes The dimension of a vector. Specifically, this means identifying key historical operational segments that lead to increased current physiological risk by calculating the attention weight of physiological states to operational trajectories; the system constructs multimodal input sequences. The mapping is represented by Q, K, and V vectors, which clearly distinguishes the roles of operational behaviors and physiological states.

[0248] Dynamic Risk Score (DRS) prediction is achieved by inputting the feature vector fused through cross-attention to the output layer of the network, connecting... Activation function, outputs real-time dynamic risk score ( ) :

[0249] Should Can be presented directly The terminal triggers tiered warnings based on the scoring gradient.

[0250] In addition, networks utilize multi-task learning ( The system outputs an auxiliary predictive metric: "Estimated Remaining Safe Operation Time" (Time-to-Desaturation). This metric quantifies the predicted vital signs (e.g., time remaining safe operation time) of the child under the current operational trend. When is it expected that the price will fall below a preset safety threshold (e.g.) This provides doctors with more timely decision support.

[0251] Step 5: Map the corrected 3D baseline model to the mixed reality (MR) field of view with sub-millimeter precision, so that it overlaps with the real-time endoscope image, i.e. the laryngoscope endoscope image, and trigger a graded warning on the mixed reality MR terminal according to the DRS scoring gradient.

[0252] This embodiment describes how to dynamically register... The model (virtual information) is fused with real-time endoscopic images (real information), and based on... The output risk score in mixed reality ( The display terminal provides real-time, tiered warnings and navigation guidance.

[0253] Geometric registration and precision assurance through virtual-real overlay During the projection process, the system utilizes the deformation-corrected output from step 3. benchmark model Combined with the high-precision six-DOF pose calculated in step 2 ,Will Each point Projected to Display terminal pixel coordinates superior.

[0254] The projection process follows a standard pinhole camera model. :

[0255]

[0256] in, This is the reverse pose transformation matrix from the laryngoscope tip to the world coordinate system. Since step 2 ensures the positioning accuracy error is less than... Furthermore, step 3 compensates for tissue deformation, thus achieving sub-millimeter-level precise overlap between the navigation lines and the glottis position, significantly improving... The accuracy of navigation maps.

[0257] To meet the stringent requirements of hand-eye coordination during surgical procedures, the overall visual feedback delay of the virtual-real overlay is... Must be controlled within Within this range. This relies on the high-frequency multi-sensor fusion in step 2 (the fusion frequency can be as high as...). The low-latency, high-update-rate pose data provided.

[0258] The system visualizes the real-time dynamic risk score (DRS) output from step 4. Integrate into Intelligent decision support is implemented in the navigation view.

[0259] A tiered early warning mechanism is adopted, based on The system triggers tiered alerts based on real-time data to remind operators to adjust their techniques.

[0260] low-risk areas ): Provides smooth green or blue navigation lines to indicate the best route.

[0261] medium-risk areas ): The system is in A yellow warning bar appears at the edge of the field of vision, accompanied by a voice prompt indicating increased risk. The navigation line color changes to yellow or orange.

[0262] High-risk areas ):exist A red warning pops up in the center of the field of vision, and high-risk anatomical structures (such as subglottic stenosis) are highlighted by color changes or flashing, forcing the operator to stop or withdraw the operation immediately.

[0263] Risk indicator visualization is in On the terminal, in addition to displaying In addition, the system also displays the following auxiliary forecasting indicators simultaneously:

[0264] Operation trajectory deviation: The current operation trajectory is indicated by a color gradient. The deviation distance of the model's optimal path planning.

[0265] Estimated remaining safe operating time (Time-to-Desaturation): This is The auxiliary output is in the form of a countdown timer (e.g., "There is also..."). Within seconds of safe operation time, blood oxygen levels will drop below [a certain value]. This visually indicates the physiological limits.

[0266] The system utilizes step 1 Anatomical semantic annotation of the benchmark model, in Key vulnerable structures (such as the vocal cords and subglottic stenosis) are geometrically highlighted in the field of view. As the laryngoscope tip approaches these structures, the virtual projection outline dynamically changes color and transparency, providing visual protection for the core targets of preventing damage to the glottis and subglottic region in infants and young children.

[0267] It should be understood that the processor in the embodiments of the present invention may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0268] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0269] It should be understood that the above-described memory is exemplary but not restrictive. For example, the memory in the embodiments of this application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM), etc. That is to say, the memory in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0270] This application also provides a computer-readable storage medium for storing computer programs.

[0271] Optionally, the computer-readable storage medium can be applied to the terminal device in the embodiments of this application, and the computer program causes the computer to execute the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0272] This application also provides a computer program product, including computer program instructions.

[0273] Optionally, the computer program product can be applied to the terminal device in the embodiments of this application, and the computer program instructions cause the computer to execute the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0274] This application also provides a computer program.

[0275] Optionally, the computer program can be applied to the vehicle autonomous driving device in the embodiments of this application. When the computer program is run on a computer, it causes the computer to execute the corresponding processes implemented by the terminal device in the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0276] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0277] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0278] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0279] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

Claims

1. An infant airway management system based on navigation and high-precision dynamic registration, characterized in that, include: The modeling and device recommendation module utilizes a transfer learning network incorporating airway hydrodynamic constraints to generate a personalized 3D airway baseline model and device recommendation parameters that satisfy physical prior conditions. The transfer learning network maps sparse image data of the child to a high-dimensional airway statistical manifold. This network employs a conditional variational autoencoder (C-VAE) architecture, receiving as input features including the child's age, weight, and Cormack-Lehane classification, along with the sparse image data. The output reconstructs a personalized model. Airway baseline model; the recommended device parameters are determined by calculating the device matching index, which is used to maximize the ratio of the outer diameter of the catheter to the narrowest point of the airway, and combined with the geometric factors of the laryngoscope blade curvature to ensure that a predetermined safe gap is maintained between the catheter and the airway; the specific calculation method of the device matching index is as follows: in, Indicates the device matching index. This indicates the recommended outer diameter of the endotracheal tube. Indicates by Airway reference model The effective diameter of the narrowest part of the child's airway was measured. Indicates the laryngoscope blade The degree of matching between the curvature of the laryngoscope and the curvature of the child's airway is used to guide the selection of a laryngoscope. This represents a weighting factor used to balance the importance of catheter size matching and laryngoscope geometry matching. Optimize objectives and find ways to achieve them. Largest duct and laryngoscope blade combination, Indicates the type of endotracheal tube to be selected. Indicates the model of the laryngoscope blade to be selected; The pose calculation module is used to simultaneously acquire endoscopic images from a laryngoscope and data from an inertial measurement unit (IMU). It performs customized enhancement processing on the endoscopic images, including red-channel weighted grayscale processing to highlight the mucosal microvascular texture, followed by adaptive histogram equalization to enhance local contrast. It extracts microscopic feature points of the airway mucosa, including mucosal microvascular texture, and calculates the real-time pose and trajectory of the laryngoscope tip. The calculation includes: using a tightly coupled fusion mechanism with an error-state extended Kalman filter (ESF) framework to calculate the real-time six-DOF pose and trajectory of the laryngoscope tip; the ESF framework addresses positioning drift during rapid intubation by fusing high-frequency IMU data with low-frequency visual data, maintaining an error state vector to estimate and compensate for the positioning drift. Its linearized evolution equation is: in This is the error state transition matrix, which explicitly describes the coupled influence of the zero bias and attitude errors of the inertial measurement unit (IMU) data on position and velocity. Represents the error state vector. middle, Represents the noise Jacobian matrix. Represents the system noise vector; The dynamic deformation field registration and compensation module is used to calculate and correct deformation errors in the 3D airway reference model caused by breathing or device compression by introducing constraints from a non-rigid deformation field based on the topological correspondence between the microscopic feature points and the 3D airway reference model. Specifically, it matches the microscopic feature points in the sparse point cloud with the static feature points using local geometric descriptors. The corresponding anatomical locations in the baseline model point set are used to establish the topological correspondence between the point sets; the non-rigid deformation field describes the displacement between the static 3D model point set and the intraoperative real-time point cloud. The non-rigid deformation field is solved by the coherent point drift (CPD) registration algorithm, thereby realizing dynamic compensation of airway soft tissue at the 3D modeling level. The risk assessment module is used to construct a multimodal spatiotemporal attention network. Taking the features of the motion trajectory and real-time physiological signals as input, it captures the causal delay features between operational behavior and physiological response through a cross-attention mechanism and outputs a real-time dynamic risk score. The early warning module is used to map the corrected 3D airway baseline model onto the mixed reality terminal's field of view, making it overlap with the laryngoscope endoscopic image, and trigger a graded early warning based on the real-time dynamic risk score.

2. The system according to claim 1, characterized in that, The pose calculation module for calculating the real-time pose and motion trajectory of the laryngoscope tip also includes: performing tight coupling fusion through a factor graph optimization framework to calculate the real-time six-degree-of-freedom pose and motion trajectory of the laryngoscope tip.

3. The system according to claim 2, characterized in that, In the pose calculation module, the factor graph optimization framework constructs a joint cost function including prior factors, pre-integration factors, and semantically weighted visual reprojection factors; the joint cost function is dynamically modulated and visual reprojection errors are processed to achieve laryngoscope tip localization.

4. The system according to claim 1, characterized in that, To achieve high-precision positioning, the pose calculation module is configured to: during visual observation updates, incorporate an extrinsic parameter transformation matrix to compensate for the physical offset between the optical center of the laryngoscope camera and the center of the inertial measurement unit (IMU), and combine it with zero-rate correction (ZUPT) logic to force zero-rate updates when the laryngoscope is in a static condition to suppress the integral drift of the IMU.

5. The system according to claim 1, characterized in that, The multimodal spatiotemporal attention network (ST-AFN) in the risk assessment module captures causal delay features through the following mechanism: A trajectory feature encoder is used to extract high-order kinematic features of the positioning trajectory output by the pose calculation module. The high-order kinematic features include at least jitter and hysteresis entropy. The cross-attention module is used to define the physiological state as Query, and the operation trajectory vector includes a key vector Key and a value vector Value. It calculates the attention weight of operation action features on changes in physiological state in order to identify key operation segments that lead to increased risk.

6. A method for infant airway management based on navigation and high-precision dynamic registration, characterized in that, include: Step 1: Using a transfer learning network that incorporates airway fluid dynamics constraints, a personalized 3D airway baseline model and recommended device parameters that satisfy the physical prior conditions are generated. The transfer learning network maps sparse image data of the child to a high-dimensional airway statistical manifold. The network employs a conditional variational autoencoder (C-VAE) architecture. Its input includes feature vectors of the child's age, weight, and Cormack-Lehane classification, along with the sparse image data. The output reconstructs a personalized... Airway baseline model; the recommended device parameters are determined by calculating the device matching index, which is used to maximize the ratio of the outer diameter of the catheter to the narrowest point of the airway, and combined with the geometric factors of the laryngoscope blade curvature to ensure that a predetermined safe gap is maintained between the catheter and the airway; the specific calculation method of the device matching index is as follows: in, Indicates the device matching index. This indicates the recommended outer diameter of the endotracheal tube. Indicates by Airway baseline model The effective diameter of the narrowest part of the child's airway was measured. Indicates the laryngoscope blade The degree of matching between the curvature of the laryngoscope and the curvature of the child's airway is used to guide the selection of a laryngoscope. This represents a weighting factor used to balance the importance of catheter size matching and laryngoscope geometry matching. Optimize objectives and find ways to achieve them. Largest duct and laryngoscope blade combination, Indicates the type of endotracheal tube to be selected. Indicates the model of the laryngoscope blade to be selected; Step 2: Simultaneously acquire endoscopic images from the laryngoscope and data from the inertial measurement unit (IMU). Perform customized enhancement processing on the endoscopic images, including red-channel weighted grayscale processing to highlight the mucosal microvascular texture. Subsequently, apply adaptive histogram equalization to the grayscale images to enhance local contrast. Extract microscopic feature points of the airway mucosa, including mucosal microvascular texture. Calculate the real-time pose and trajectory of the laryngoscope tip. Perform tight-coupled fusion using an error-state extended Kalman filter framework to calculate the real-time six-DOF pose and trajectory of the laryngoscope tip. The error-state extended Kalman filter framework solves the positioning drift problem under rapid intubation by fusing high-frequency IMU data and low-frequency visual data. To estimate and compensate for the positioning drift, the error-state extended Kalman filter framework maintains an error state vector. Its linearized evolution equation is: in This is the error state transition matrix, which explicitly describes the coupled influence of the zero bias and attitude errors of the inertial measurement unit (IMU) data on position and velocity. Represents the error state vector. middle, Represents the noise Jacobian matrix. Represents the system noise vector; Step 3: Based on the topological correspondence between the microscopic feature points and the 3D airway reference model, a non-rigid deformation field constraint is introduced to calculate and correct the deformation error in the 3D airway reference model caused by breathing or device compression; wherein, the microscopic feature points in the sparse point cloud of microscopic feature points and the static feature points are matched by local geometric descriptors. The corresponding anatomical locations in the baseline model point set are used to establish the topological correspondence between the point sets; the non-rigid deformation field describes the displacement between the static 3D model point set and the intraoperative real-time point cloud. The non-rigid deformation field is solved by the coherent point drift (CPD) registration algorithm, thereby realizing dynamic compensation of airway soft tissue at the 3D modeling level. Step 4: Construct a multimodal spatiotemporal attention network. Using the features of the motion trajectory and real-time physiological signals as input, capture the causal delay features between operational behavior and physiological response through a cross-attention mechanism, and output a real-time dynamic risk score. Step 5: Map the corrected 3D airway baseline model onto the mixed reality terminal's field of view, making it overlap with the laryngoscope endoscopic image, and trigger a graded warning based on the real-time dynamic risk score.

Images