A laryngoscope intubation posture navigation method and system based on tracheal contour recognition

Abstract

The present application relates to the technical field of medical image processing, in particular to a laryngoscope intubation posture navigation method and system based on trachea contour identification, comprising: acquiring an RGB-D image sequence by a multi-spectral depth camera integrated at the end of a laryngoscope, segmenting a trachea region and extracting a depth label image by using a U-Net network; generating a dynamic trachea cartilage ring contour point cloud by using a double-threshold Hessian filter and a curvature gradient segmentation; realizing multi-frame point cloud deformation compensation based on a non-rigid ICP, and fitting a trachea axis direction vector and an entrance center coordinate by using a principal component analysis method; calculating an offset angle and a depth distance in combination with real-time pose data of the laryngoscope tube end, generating a dynamic guide arrow, and triggering an audible and light warning signal when approaching the trachea entrance. The present application realizes three-dimensional navigation guidance and safety distance warning during intubation, improves operation precision and safety, and is suitable for clinical anesthesia, emergency intubation and airway management scenarios.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method and system for laryngoscopy intubation posture navigation based on tracheal contour recognition. Background Technology

[0002] In clinical anesthesia and emergency intubation procedures, laryngoscopy-assisted endotracheal intubation is one of the most commonly used methods for establishing an artificial airway. Traditional procedures rely on the physician's subjective judgment and experience in determining the location of the tracheal inlet. During intubation, it is crucial to accurately identify tracheal anatomy, especially the cartilaginous rings and glottis, and to control the direction and depth of laryngoscope advancement to avoid accidental insertion into the esophagus or damage to the laryngeal mucosa. With the development of endoscopic imaging technology and augmented reality devices, image recognition-based intelligent intubation assistance systems have gradually become a research hotspot, aiming to improve intubation success rates and reduce complications through visual enhancement and path guidance technologies.

[0003] However, existing technologies still have significant shortcomings in real-time intubation navigation: on the one hand, single-channel imaging systems struggle to cope with recognition errors caused by blurred mucosal textures, uneven lighting, or tissue deformation, and lack a deep understanding of tracheal structures and the ability to model three-dimensional space; on the other hand, the intubation path lacks visual navigation and safety warning mechanisms, and doctors cannot obtain real-time posture deviation and depth feedback, easily leading to accidental intubation and injury. Although some methods use deep learning to assist in recognition, they fail to integrate multimodal perception data to construct a stable three-dimensional tracheal model and also lack guidance mechanisms that are linked to the actual intubation posture. Summary of the Invention

[0004] This invention provides a method and system for laryngoscopy intubation posture navigation based on tracheal contour recognition. This method and system combines structural recognition accuracy, spatiotemporal stability, and dynamic feedback capability, thereby improving the intelligence level and safety assurance of clinical operations.

[0005] A laryngoscope intubation posture navigation method based on tracheal contour recognition includes the following steps: S1: Simultaneously acquire RGB-D image sequences of the laryngeal region using a multispectral depth camera at the end of the laryngoscope, and output tracheal region images with depth labels; S2: Adaptive bone ring enhancement processing is performed on the tracheal region image output by S1. The tracheal cartilage ring ripples are segmented based on curvature gradient to generate dynamic tracheal cartilage ring contour point cloud. S3: Input the dynamic tracheal cartilage ring contour point cloud output by S2 into the deformation compensation model to fit the axial direction vector of the trachea in three-dimensional space and the coordinates of the inlet center. S4: Based on the axis direction vector output by S3 and the inlet center coordinates, calculate the real-time offset angle and depth distance between the laryngoscope tube end and the tracheal axis, and generate dynamic guide arrows and safety distance warning signals.

[0006] Optionally, S1 includes: S11: Simultaneously emits near-infrared 850nm band and visible light 530nm band through a multispectral depth camera integrated at the end of the laryngoscope, acquires the original multispectral depth image sequence of the laryngeal region at a frame rate of ≥25fps, and outputs the original multispectral depth image sequence. S12: Perform tracheal region segmentation based on U-Net convolutional neural network on the original multispectral depth image sequence output by S11, and extract mucosal texture and cartilage ring position features using the depth feature fusion layer in the pre-trained model to generate a tracheal region mask map. S13: Perform pixel-level fusion of the tracheal region mask image output from S12 with the depth channels in the original multispectral depth image sequence, retaining the depth information of all pixels within the mask area, and generate the tracheal region image with depth label.

[0007] Optionally, the adaptive bone ring enhancement processing employs a dual-threshold Hessian matrix filter, with the first threshold extracting the cartilage ring ridge line and the second threshold separating respiratory motion artifacts.

[0008] Optionally, the curvature gradient segmentation specifically involves: calculating the local curvature derivative for ridge pixel points, and retaining a continuous set of points with a curvature change rate > 0.35 to form a contour point cloud.

[0009] Optionally, S2 includes: S21: Perform dual-threshold Hessian matrix filtering on the depth-labeled tracheal region image output by S1. The first threshold of 0.25 is used to extract the cricoid ridge features, and the second threshold of 0.05 is used to filter out respiratory motion artifacts, generating a tracheal cricoid ridge image. S22: Calculate the local curvature derivative pixel by pixel for the tracheal cartilage ring ridge image output by S21, filter pixels with curvature change rate > 0.35, and merge continuous point sets with spatial distance < 0.2 mm based on deep connected component analysis to generate candidate point cloud of tracheal cartilage rings. S23: Perform motion consistency verification between the candidate point cloud of the tracheal cartilage rings output by S22 and the historical point cloud of 3 consecutive frames, remove outliers with displacement variance > 1.5mm, and integrate them to form the dynamic tracheal cartilage ring contour point cloud.

[0010] Optionally, the deformation compensation model of S3 aligns the point cloud of consecutive frames using a non-rigid ICP algorithm to compensate for the contour distortion caused by breathing deformation.

[0011] Optionally, the axis direction vector is fitted to the compensated point cloud using principal component analysis, and the direction of the first principal component is taken as the axis direction vector.

[0012] Optionally, S3 includes: S31: Input the dynamic tracheal cartilage ring contour point cloud output by S2 into the deformation compensation model, use the non-rigid ICP algorithm to perform spatiotemporal registration of the point cloud for 5 consecutive frames, compensate for respiratory motion deformation by minimizing the energy function of the point cloud distance between adjacent frames, and generate the tracheal contour point cloud after deformation compensation. S32: Perform principal component analysis on the tracheal contour point cloud output by S31 after deformation compensation, calculate the eigenvector corresponding to the largest eigenvalue of the point cloud covariance matrix, and obtain the axis direction vector after normalization. S33: In the tracheal contour point cloud after deformation compensation output by S31, search for the minimum Z coordinate point along the axial direction vector obtained by S32, take the centroid of the point cloud in a spherical domain with a radius of 3mm as the center, and output the coordinates of the inlet center.

[0013] Optionally, S4 includes: S41: Based on the axis direction vector and inlet center coordinates output by S3, and combined with the real-time pose data of the laryngoscope tube end, calculate the spatial angle between the laryngoscope tube end direction vector and the tracheal axis direction vector as the real-time offset angle, and at the same time calculate the Euclidean distance between the laryngoscope tube end and the inlet center coordinates as the depth distance. S42: Maps the real-time offset angle output by S41 to the arrow deflection angle in the augmented reality display. The arrow length shortens proportionally as the depth distance decreases, and the color gradually changes from blue when the depth distance is >20mm to red when it is <10mm, forming a dynamic guide arrow. S43: When the depth distance output by S41 is less than 10mm, the sound and light generator is activated to output a buzzer signal whose pulse frequency is negatively correlated with the depth distance. At the same time, the bezel of the augmented reality display is driven to flash red light to generate a safe distance warning signal.

[0014] A laryngoscope intubation posture navigation system based on tracheal contour recognition, used to implement the aforementioned laryngoscope intubation posture navigation method based on tracheal contour recognition, includes the following modules: Multispectral depth image acquisition module: used to simultaneously emit near-infrared and visible light light sources through a multispectral depth camera integrated at the end of the laryngoscope, acquire RGB-D image sequences of the laryngeal region at a frame rate of not less than 25 frames / second, and output tracheal region images with depth labels; Adaptive bone ring enhancement module: used to perform dual-threshold Hessian matrix filtering on the depth-labeled tracheal region image, extract the tracheal cartilage ring ridge image, and combine curvature gradient analysis and depth connected component filtering to output dynamic tracheal cartilage ring contour point cloud. 3D trachea modeling module: used to receive the dynamic tracheal cartilage ring contour point cloud, perform deformation compensation of continuous point cloud based on non-rigid ICP algorithm, fit the axial direction vector of trachea using principal component analysis method, and calculate the inlet center coordinates; Navigation parameter calculation module: used to receive laryngoscope tube end pose data in real time, and combine the axis direction vector and inlet center coordinates to calculate the offset angle and three-dimensional Euclidean distance between the laryngoscope tube end direction and the tracheal axis, and generate navigation parameters; Augmented Reality Guidance Module: Used to map the navigation parameters into visual guidance arrows, control their direction, length and color to dynamically change, and realize tube insertion path guidance; Safety warning trigger module: When the Euclidean distance is less than a preset threshold, it outputs a buzzer signal that is negatively correlated with the distance and drives the augmented reality display to flash a red border to realize a safety distance warning prompt.

[0015] The beneficial effects of this invention are: This invention employs a multispectral depth camera integrated into the end of a laryngoscope, fusing 850nm near-infrared and 530nm visible light images to acquire RGB-D image sequences in real time at a frame rate of ≥25fps. A U-Net convolutional neural network is introduced for tracheal region segmentation, combined with a bi-branch feature extraction structure and dynamic deformation enhancement strategy in a pre-trained model, effectively extracting the tracheal cartilage ring structure and mucosal texture, achieving accurate annotation of depth information. This method significantly improves the accuracy and robustness of structural recognition under complex lighting and respiratory turbulence conditions in clinical practice, providing high-quality input for subsequent navigation.

[0016] This invention employs dual-threshold Hessian matrix filtering to enhance the tracheal cartilage ring ridge line, then extracts candidate point clouds through curvature gradient segmentation and point cloud connectivity analysis, and combines a non-rigid ICP algorithm for point cloud deformation compensation across 5 frames to obtain a stable and reliable tracheal contour point cloud. Simultaneously, principal component analysis (PCA) is used to extract the tracheal axis direction vector, and the inlet center coordinates are calculated in the neighborhood of the minimum projection point in the Z-axis, effectively overcoming posture shifts and structural misalignments caused by respiratory motion, ensuring the continuity and spatial accuracy of the intubation path modeling.

[0017] This invention, based on real-time acquired direction vector and position coordinates of the laryngoscope tip, calculates its angle and depth distance with the tracheal axis, generating corresponding dynamic guide arrows. It automatically maps the offset angle, adjusts the arrow length, and implements color gradients to guide the operator in real-time correction of the intubation direction. Simultaneously, when the depth distance is less than 10mm, the system triggers an audible and visual alarm, emitting a beeping sound with a frequency increasing with distance and flashing the AR display border. This effectively constructs a multi-dimensional warning mechanism combining visualization and hearing, significantly improving intubation safety and operational error tolerance. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the system flow according to an embodiment of the present invention. Detailed Implementation

[0020] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0021] It should be noted that the use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.

[0022] Generally, terms can be understood at least partly from their use in context. For example, depending at least partly on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in a singular sense, or a combination of features, structures, or characteristics in a plural sense. Additionally, the term "based on" can be understood not necessarily to convey an exclusive set of factors, but rather, alternatively, depending at least partly on the context, to allow for the presence of other factors that are not necessarily explicitly described.

[0023] like Figure 1 As shown, a laryngoscope intubation posture navigation method based on tracheal contour recognition includes the following steps: S1: Simultaneously acquire RGB-D image sequences of the laryngeal region using a multispectral depth camera at the tip of the laryngoscope, and output tracheal region images with depth labels, specifically: S11, Multispectral Depth Image Acquisition: Image acquisition is achieved through a multispectral depth camera integrated into the laryngoscope tip. The camera is capable of simultaneously emitting near-infrared 850nm and visible light 530nm wavelengths, employing a dual-band synchronous imaging method to enhance the identification of structural differences in tracheal tissue. Specifically, the multispectral depth camera acquires images of the laryngeal region in real time at a frame rate of no less than 25fps (frames / second), forming a multispectral image sequence with spatial depth information. The 850nm near-infrared band is used to enhance the extraction of the reflectivity of tracheal cartilage in soft tissue structures, while the 530nm visible light band is used to capture tracheal mucosal texture and color information. Together, they constitute the original multispectral depth image sequence with an RGB-D structure, which is then output to the subsequent image segmentation module.

[0024] S12, Tracheal Region Segmentation Based on U-Net Convolutional Neural Network: The original multispectral depth image sequence is processed by a U-Net convolutional neural network to perform pixel-level segmentation of the tracheal region. This U-Net model is an encoder-decoder semantic segmentation network, and its pre-trained model includes the following components: Tracheal cartilage ring annotation dataset: This dataset contains 5000 annotated samples. Each sample consists of a sequence of multispectral depth images acquired during surgery, along with a corresponding tracheal region mask. The mask uses polygon vertices to manually annotate the wavy boundaries of the tracheal cartilage rings to ensure segmentation accuracy and boundary coherence.

[0025] Dual-branch feature extraction network: The first branch is the visible light branch, which uses the ResNet-34 network structure as the backbone to extract high-level texture features of the tracheal mucosa surface from RGB images and generate a high-level visible light feature map. The second branch is the depth branch, which uses the PointNet++ network structure as the backbone to perform spatial point cloud processing on the depth channel of the original image, extract the three-dimensional structure features, and output the depth point cloud feature vector.

[0026] Deep Feature Fusion Layer: The visible light high-level feature maps output from the two branches are concatenated with the depth point cloud feature vectors along the channel dimension. After convolution with a 3×3 convolution kernel, a fused feature tensor is generated. The fused feature tensor is then passed through an activation function and upsampling operations to output a trachea region probability heatmap with the same size as the input image, which is used for subsequent mask generation.

[0027] Dynamic deformation compensation training strategy: In order to improve the robustness of the model to soft tissue displacement caused by breathing and laryngeal movement, random motion blur and affine transformation perturbation are introduced into the image sequence during the training phase to simulate the visual changes during laryngoscopy intubation in clinical practice, thereby improving the recognition accuracy and stability of the model under actual operating conditions.

[0028] After processing by the above model, the output trachea region probability heatmap is further processed by threshold segmentation to generate a trachea region mask map. This mask map contains a binary label region of the same size as the original image, marking the region where the trachea is located. S13, fusing the mask image and depth image to generate a depth-labeled image: The tracheal region mask image output from S12 is fused pixel-wise with the depth channels of the original multispectral depth image sequence obtained in S11. Specifically, for each frame, its depth channels (usually in 16-bit grayscale matrix form) are extracted. The mask image is used as a filter template, retaining only the depth information of the corresponding pixels within the mask area and masking redundant depth data from non-tracheal regions. The fused image forms a depth-labeled tracheal region image containing accurate depth information, which is the final output of S1. This image preserves the boundary structure and depth distribution of the tracheal tissue, providing basic data support for subsequent steps such as bony ring enhancement, 3D contour point cloud extraction, and pose offset calculation.

[0029] During this process, the entire S1 data stream is a continuous real-time processing flow. The system can achieve minimum inter-frame latency (less than 100ms) between image acquisition, segmentation and fusion, meeting the dual requirements of real-time performance and accuracy during laryngoscope intubation navigation.

[0030] S2: Adaptive bone ring enhancement processing is performed on the tracheal region image output by S1. Based on the curvature gradient segmentation of the tracheal cartilage ring ripples, a dynamic tracheal cartilage ring contour point cloud is generated, specifically: S21, Adaptive Bone Ring Enhancement Processing: For the depth-labeled tracheal region image output by S1, perform image enhancement processing based on dual-threshold Hessian matrix filtering to highlight the ridge structure of the tracheal cartilage rings and suppress background disturbances caused by respiration.

[0031] The specific method is as follows: First, perform a two-dimensional Hessian matrix calculation on each frame of the image to obtain the second derivative response of each pixel in its local neighborhood; The eigenvalues ​​of the calculated Hessian matrix are analyzed to extract regions with linear enhanced responses; Two response thresholds are set: the first threshold is 0.25, which is used to identify the main ridge structure of the cartilage ring and retain pixels with response values ​​greater than this threshold; the second threshold is 0.05, which is used to exclude weak response areas caused by slight respiratory movements and remove pixels with response values ​​less than this value.

[0032] The image after dual-threshold filtering is the tracheal cartilage cricoid ridge image, which retains a high-response curve with anatomical features while effectively eliminating motion artifacts. This image is represented in grayscale, with grayscale values ​​corresponding to Hessian response intensity, for subsequent curvature analysis.

[0033] S22, Curvature gradient segmentation: Calculate the local curvature derivative pixel by pixel in the tracheal cartilage ring ridge image to identify structural points with significant curvature changes, thereby extracting candidate boundary points of the tracheal cartilage ring.

[0034] The specific processing procedure includes: In the ridge image, the rate of curvature change (curvature gradient) of each pixel is calculated in a 3×3 pixel sliding window. Pixels with a curvature change rate greater than 0.35 are selected as having edge ripples or structural abrupt changes in the cartilage ring. The selected pixels are mapped into a set of three-dimensional coordinate points in the image based on their three-dimensional depth information; Based on this 3D point set, deep connected component analysis is performed: points with a Euclidean distance of less than 0.2mm in 3D space are clustered and merged into several continuous structural units to eliminate isolated noise points and falsely detected edge points.

[0035] The final output is a candidate point cloud of tracheal cartilage rings. Each point cloud unit represents a continuous cartilage structure segment with boundary features and spatial geometric consistency.

[0036] S23, Dynamic Contour Generation: To improve the stability and robustness of tracheal cartilage ring contour extraction, an inter-frame point cloud motion consistency verification mechanism is used to dynamically screen candidate point clouds.

[0037] The specific method is as follows: Obtain the candidate point cloud of the tracheal cartilage ring in the current frame, and retrieve the historical point cloud data of the three consecutive frames that are adjacent to it in time. For each candidate point, calculate the displacement variance of its corresponding spatial position in the current frame and the previous 3 frames; this variance represents the motion stability of the point in a short time. Set a threshold: if the displacement variance of a certain point is greater than 1.5mm, then the point is considered to be an outlier caused by motion anomalies or artifact interference, and it is removed. Aggregation operations are performed on the retained points to integrate them into a dynamic tracheal cartilage ring contour point cloud that is structurally continuous, spatially smooth, and stable across frames.

[0038] This point cloud not only describes the spatial shape of the tracheal cartilage rings in the current frame, but also has temporal stability, making it the core input data for subsequent three-dimensional axis fitting and attitude offset calculations. S3: Input the dynamic tracheal cartilage ring contour point cloud output from S2 into the deformation compensation model, and fit the tracheal axis direction vector and inlet center coordinates in three-dimensional space, specifically: S31, Dynamic Point Cloud Deformation Compensation: The dynamic tracheal cartilage ring contour point cloud output from S2 is input into the deformation compensation model. A non-rigid ICP algorithm is used to perform spatiotemporal registration of the point cloud for 5 consecutive frames. Respiratory motion deformation is compensated by minimizing the energy function of the point cloud distance between adjacent frames, generating a deformation-compensated tracheal contour point cloud. The specific method is as follows: First, a time window point cloud sequence is constructed, consisting of the current frame and the two frames immediately before and after it, for a total of five frames, representing the dynamic contour point clouds of the tracheal cartilage rings. This sequence is denoted as […]. ; The sequence is then sequentially registered frame-by-frame using the non-rigid ICP algorithm. This algorithm iterates through the transformation matrix between each pair of adjacent frames to find transformation parameters that minimize the following distance-energy function: ; in, To establish matching point pairs between the current frame and the next frame. It is a locally rigid rotation matrix. It is a translation vector. The coefficients for the deformation smoothing regularization term control the non-rigidity. The first term is the registration error term. The second term is a local deformation constraint term, which suppresses local structural tearing; Through the above process, all 5 frames of point clouds are registered and deformed to minimize the structural error between point clouds and obtain a time-consistent point cloud set under a unified coordinate system.

[0039] Ultimately, the output is a point cloud of the tracheal contour after deformation compensation. This eliminates displacement errors caused by respiratory movements or changes in perspective, preserves the true tracheal structure information, and provides accurate data support for subsequent orientation fitting and entrance localization.

[0040] S32, Tracheal Axis Direction Fitting: Spatial structural direction analysis is performed on the tracheal contour point cloud after deformation compensation. Principal component analysis is used to fit its main axis directions. The specific steps are as follows: The deformed point cloud set is represented as a three-dimensional coordinate set. ; Calculate the covariance matrix of this point set. Its definition is as follows: ; in, The mean center coordinates of the point cloud.

[0041] For covariance matrix Perform eigenvalue decomposition to obtain eigenvalue-eigenvector pairs. , where the largest eigenvalue The corresponding eigenvector Indicates the main direction of the point cloud; For this feature vector After normalization, a unit direction vector is obtained, which is defined as the axis direction vector in this invention. .

[0042] This direction vector represents the main extension direction of the tracheal structure in three-dimensional space, serving as the directional reference for intubation path calibration.

[0043] S33, Inlet center coordinate calculation: After obtaining the axis direction vector, in order to achieve accurate registration between the laryngoscope tube and the tracheal axis, it is necessary to identify the inlet center coordinate, that is, the spatial position center of the tracheal opening.

[0044] The specific calculation steps are as follows: In the tracheal contour point cloud after deformation compensation, the projection is sorted according to the axial direction vector obtained in S32; let this direction vector be a unit vector. Then each point cloud The projection onto the direction vector is: ; Search for the point in the point cloud with the smallest projection value (i.e., Z-coordinate). This point is considered to be located in the anterior region of the tracheal opening; by Centered on a sphere with a radius of 3 mm, extract all point clouds falling within this sphere to form a subset. ; Calculate the subset The geometric centroid coordinates, used as the entrance center coordinates in this invention, are defined as follows: ; The final output of the inlet center coordinates and the axis direction vector together constitute the attitude reference for three-dimensional cannulation navigation, which is used for subsequent navigation deviation calculation and guidance command generation.

[0045] S4: Based on the axial direction vector output by S3 and the inlet center coordinates, calculate the real-time offset angle and depth distance between the laryngoscope tube tip and the tracheal axis, and generate a dynamic guide arrow and a safety distance warning signal, specifically: S41, Real-time calculation of navigation parameters: Based on the axis direction vector output by S3 and the inlet center coordinates, combined with the real-time pose data of the laryngoscope tube end, the spatial angle between the laryngoscope tube end direction vector and the tracheal axis direction vector is calculated as the real-time offset angle. At the same time, the Euclidean distance between the laryngoscope tube end and the inlet center coordinates is calculated as the depth distance. The specific method is as follows: 1. Laryngoscope tube pose acquisition: The spatial orientation vector is acquired in real time using an attitude tracking unit (such as a six-axis IMU or a vision-inertial synchronization module) mounted on the front end of the laryngoscope. and three-dimensional position coordinates ; 2. Offset Angle Calculation: Calculate the laryngoscope direction vector. Vector of direction relative to tracheal axis The included angle The formula is as follows: ; in," " represents the vector dot product, For modulo length calculation.

[0046] 3. Depth distance calculation: Calculate the position of the laryngoscope tip. coordinates of the tracheal inlet center The three-dimensional Euclidean distance between them The formula is: The final output parameters include: Real-time offset angle This is used to assess the degree of orientation registration. Depth Distance It is used to determine the proximity of the cannula.

[0047] S42. Dynamic Guide Arrow Generation: To provide the operator with visual navigation guidance, the system generates guide arrows based on the aforementioned real-time offset angles. and depth distance To generate continuously dynamically changing guide arrows in an augmented reality display, the specific steps are as follows: 1. Direction mapping and deflection angle calculation: [This section appears to be incomplete and requires further context.] Mapped to the arrow deflection angle in augmented reality images The calculation method is as follows: ; in To display the magnification factor, used to improve the sensitivity of visual guidance.

[0048] 2. Arrow length adjustment: Set the initial reference arrow length According to depth distance Scaling arrow length ,satisfy: .

[0049] 3. Arrow color gradient control: When As the arrow decreases, it shortens accordingly, indicating that the target area for cannulation is approaching. The arrow color is based on... The value changes dynamically, according to the following rules: The arrow is blue; The color gradually changes from blue to red; The arrow turns red, indicating a risky operation.

[0050] 4. Augmented Reality Rendering: The arrows described above are rendered in real time on an augmented reality display (such as a head-mounted display or a transparent OLED screen), with the tail of the arrow anchored at the end of the laryngoscope and the head pointing towards... Direction, dynamic feedback of angle and distance.

[0051] S43, Safety warning signal triggered: When the tip of the laryngoscope gets too close to the tracheal inlet area, to prevent tissue damage and misinsertion, the system will activate based on the depth and distance. Activate the multimodal early warning mechanism.

[0052] The specific method is as follows: 1. Early warning judgment threshold: when When the laryngoscope approaches a high-risk area, an early warning system is triggered.

[0053] 2. Buzzer Signal Generation: Controls the audible and visual alarm to emit a buzzer sound whose frequency is negatively correlated with distance. The calculation formula is as follows: ;in: .

[0054] The deeper the insertion, the higher the beeping frequency, enhancing the sense of urgency.

[0055] 3. Blinking indicator: At the same time, the border of the augmented reality display interface will flash a bright red border at a frequency of 1Hz to continuously remind the operator to slow down the movement and pay attention to adjusting the direction.

[0056] Ultimately, the safe distance warning signal is output through both sound and light channels to ensure immediate and strong feedback safety in high-risk areas.

[0057] like Figure 2 As shown, a laryngoscope intubation posture navigation system based on tracheal contour recognition is used to implement the aforementioned laryngoscope intubation posture navigation method based on tracheal contour recognition, and includes the following modules: Multispectral depth image acquisition module: used to simultaneously emit near-infrared and visible light light sources through a multispectral depth camera integrated at the end of the laryngoscope, acquire RGB-D image sequences of the laryngeal region at a frame rate of not less than 25 frames / second, and output tracheal region images with depth labels; Adaptive Bone Ring Enhancement Module: This module performs dual-threshold Hessian matrix filtering on tracheal region images with depth labels to extract tracheal cartilage ring ridge images. It then combines curvature gradient analysis and depth connected component filtering to output dynamic tracheal cartilage ring contour point clouds. 3D trachea modeling module: used to receive dynamic tracheal cartilage ring contour point cloud, perform deformation compensation of continuous point cloud based on non-rigid ICP algorithm, fit the axial direction vector of trachea using principal component analysis method, and calculate the coordinates of the inlet center. Navigation parameter calculation module: used to receive laryngoscope tube end pose data in real time, and combine the axis direction vector and the inlet center coordinates to calculate the offset angle and three-dimensional Euclidean distance between the laryngoscope tube end direction and the tracheal axis, and generate navigation parameters; Augmented Reality Guidance Module: Used to map navigation parameters into visual guide arrows, controlling their direction, length, and color to dynamically change, thereby guiding the insertion path; Safety warning trigger module: When the Euclidean distance is less than a preset threshold, it outputs a buzzer signal that is negatively correlated with the distance and drives the augmented reality display to flash a red border to realize a safety distance warning prompt.

[0058] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0059] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A laryngoscope intubation posture navigation method based on tracheal contour recognition, characterized in that, Includes the following steps: S1: Simultaneously acquire RGB-D image sequences of the laryngeal region using a multispectral depth camera at the end of the laryngoscope, and output tracheal region images with depth labels; S2: Adaptive bone ring enhancement processing is performed on the tracheal region image output by S1. The tracheal cartilage ring ripples are segmented based on curvature gradient to generate dynamic tracheal cartilage ring contour point cloud. S3: Input the dynamic tracheal cartilage ring contour point cloud output by S2 into the deformation compensation model to fit the axial direction vector of the trachea in three-dimensional space and the coordinates of the inlet center. S4: Based on the axis direction vector output by S3 and the inlet center coordinates, calculate the real-time offset angle and depth distance between the laryngoscope tube end and the tracheal axis, and generate dynamic guide arrows and safety distance warning signals.

2. The laryngoscope intubation posture navigation method based on tracheal contour recognition according to claim 1, characterized in that, S1 includes: S11: Simultaneously emits near-infrared 850nm band and visible light 530nm band through a multispectral depth camera integrated at the end of the laryngoscope, acquires the original multispectral depth image sequence of the laryngeal region at a frame rate of ≥25fps, and outputs the original multispectral depth image sequence. S12: Perform tracheal region segmentation based on U-Net convolutional neural network on the original multispectral depth image sequence output by S11, and extract mucosal texture and cartilage ring position features using the depth feature fusion layer in the pre-trained model to generate a tracheal region mask map. S13: Perform pixel-level fusion of the tracheal region mask image output from S12 with the depth channels in the original multispectral depth image sequence, retaining the depth information of all pixels within the mask area, and generate the tracheal region image with depth label.

3. The laryngoscope intubation posture navigation method based on tracheal contour recognition according to claim 2, characterized in that, The adaptive bone ring enhancement process uses a dual-threshold Hessian matrix filter. The first threshold extracts the cartilage ring ridge line, and the second threshold separates respiratory motion artifacts.

4. The laryngoscope intubation posture navigation method based on tracheal contour recognition according to claim 3, characterized in that, The curvature gradient segmentation specifically involves calculating the local curvature derivative for ridge pixel points and retaining continuous point sets with a curvature change rate > 0.35 to form a contour point cloud.

5. The laryngoscope intubation posture navigation method based on tracheal contour recognition according to claim 4, characterized in that, S2 includes: S21: Perform dual-threshold Hessian matrix filtering on the depth-labeled tracheal region image output by S1. The first threshold of 0.25 is used to extract the cricoid ridge features, and the second threshold of 0.05 is used to filter out respiratory motion artifacts, generating a tracheal cricoid ridge image. S22: Calculate the local curvature derivative pixel by pixel for the tracheal cartilage ring ridge image output by S21, filter pixels with curvature change rate > 0.35, and merge continuous point sets with spatial distance < 0.2 mm based on deep connected component analysis to generate candidate point cloud of tracheal cartilage rings. S23: Perform motion consistency verification between the candidate point cloud of the tracheal cartilage rings output by S22 and the historical point cloud of 3 consecutive frames, remove outliers with displacement variance > 1.5mm, and integrate them to form the dynamic tracheal cartilage ring contour point cloud.

6. The laryngoscope intubation posture navigation method based on tracheal contour recognition according to claim 5, characterized in that, The deformation compensation model of S3 aligns the point cloud of consecutive frames using a non-rigid ICP algorithm to compensate for the contour distortion caused by breathing deformation.

7. A laryngoscope intubation posture navigation method based on tracheal contour recognition according to claim 6, characterized in that, The axis direction vector is obtained by using principal component analysis to perform dimensionality reduction fitting on the compensated point cloud, and the direction of the first principal component is taken as the axis direction vector.

8. The laryngoscope intubation posture navigation method based on tracheal contour recognition according to claim 7, characterized in that, S3 includes: S31: Input the dynamic tracheal cartilage ring contour point cloud output by S2 into the deformation compensation model, use the non-rigid ICP algorithm to perform spatiotemporal registration of the point cloud for 5 consecutive frames, compensate for respiratory motion deformation by minimizing the energy function of the point cloud distance between adjacent frames, and generate the tracheal contour point cloud after deformation compensation. S32: Perform principal component analysis on the tracheal contour point cloud output by S31 after deformation compensation, calculate the eigenvector corresponding to the largest eigenvalue of the point cloud covariance matrix, and obtain the axis direction vector after normalization. S33: In the tracheal contour point cloud after deformation compensation output by S31, search for the minimum Z coordinate point along the axial direction vector obtained by S32, take the centroid of the point cloud in a spherical domain with a radius of 3mm as the center, and output the coordinates of the inlet center.

9. A laryngoscope intubation posture navigation method based on tracheal contour recognition according to claim 8, characterized in that, S4 includes: S41: Based on the axis direction vector and inlet center coordinates output by S3, and combined with the real-time pose data of the laryngoscope tube end, calculate the spatial angle between the laryngoscope tube end direction vector and the tracheal axis direction vector as the real-time offset angle, and at the same time calculate the Euclidean distance between the laryngoscope tube end and the inlet center coordinates as the depth distance. S42: Maps the real-time offset angle output by S41 to the arrow deflection angle in the augmented reality display. The arrow length shortens proportionally as the depth distance decreases, and the color gradually changes from blue when the depth distance is >20mm to red when it is <10mm, forming a dynamic guide arrow. S43: When the depth distance output by S41 is less than 10mm, the sound and light generator is activated to output a buzzer signal whose pulse frequency is negatively correlated with the depth distance. At the same time, the bezel of the augmented reality display is driven to flash red light to generate a safe distance warning signal.

10. A laryngoscope intubation posture navigation system based on tracheal contour recognition, used to implement the laryngoscope intubation posture navigation method based on tracheal contour recognition as described in any one of claims 1-9, characterized in that, Includes the following modules: Multispectral depth image acquisition module: used to simultaneously emit near-infrared and visible light light sources through a multispectral depth camera integrated at the end of the laryngoscope, acquire RGB-D image sequences of the laryngeal region at a frame rate of not less than 25 frames / second, and output tracheal region images with depth labels; Adaptive bone ring enhancement module: used to perform dual-threshold Hessian matrix filtering on the depth-labeled tracheal region image, extract the tracheal cartilage ring ridge image, and combine curvature gradient analysis and depth connected component filtering to output dynamic tracheal cartilage ring contour point cloud. 3D trachea modeling module: used to receive the dynamic tracheal cartilage ring contour point cloud, perform deformation compensation of continuous point cloud based on non-rigid ICP algorithm, fit the axial direction vector of trachea using principal component analysis method, and calculate the inlet center coordinates; Navigation parameter calculation module: used to receive laryngoscope tube end pose data in real time, and combine the axis direction vector and inlet center coordinates to calculate the offset angle and three-dimensional Euclidean distance between the laryngoscope tube end direction and the tracheal axis, and generate navigation parameters; Augmented Reality Guidance Module: Used to map the navigation parameters into visual guidance arrows, control their direction, length and color to dynamically change, and realize tube insertion path guidance; Safety warning trigger module: When the Euclidean distance is less than a preset threshold, it outputs a buzzer signal that is negatively correlated with the distance and drives the augmented reality display to flash a red border to realize a safety distance warning prompt.

Images