Tracheotomy anatomical structure image recognition method and system

Abstract

The invention relates to the technical field of medical instruments, and discloses a tracheotomy anatomical structure image recognition method and system, and the method comprises the steps: obtaining real-time visual image data and spatial positioning data of an endoscope, carrying out the time synchronization, carrying out the mode recognition of the spatial positioning data, and locking the remarkable anatomical features in the real-time visual image data, and according to the characteristics and the spatial positioning data, calculating a geometric transformation relationship between an endoscope visual coordinate system and a spatial positioning coordinate system, and finally fusing the data and displaying anatomical structure information and a surgical tool position in real time. The method can effectively solve the problem that in the prior art, an endoscope is irregularly deformed in the narrow trachea with physiological bending of a patient.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a method and system for image recognition of anatomical structures in tracheotomy surgery. Background Technology

[0002] In modern surgery, tracheotomy, especially percutaneous tracheotomy, is a procedure requiring extremely high precision. This is because vital blood vessels and nerves are distributed around the trachea, and even the slightest deviation in positioning can lead to serious complications. To improve the safety and accuracy of the surgery, current technological trends are towards introducing advanced navigation and identification methods, such as combining the intuitive visual information from endoscopy with the precise spatial positioning data from electromagnetic navigation, aiming to identify the anatomical structures within the trachea in real time and accurately during the operation, and guide surgical instruments to their predetermined positions. However, in actual surgical settings, even with these advanced technologies, some unexpected challenges may arise, which often affect the precision of the surgery and the safety of the patient. Summary of the Invention

[0003] This invention provides a method and system for image recognition of anatomical structures in tracheotomy, aiming to solve the technical problem that in tracheotomy, the deformation of the endoscope under force inside the trachea causes positioning deviation between the image coordinate system and the spatial coordinate system, which in turn affects the accuracy of the surgery and the safety of the patient.

[0004] In a first aspect, to solve the above-mentioned technical problems, the present invention provides a method for image recognition of anatomical structures in tracheotomy surgery, comprising: Acquire real-time visual image data and spatial positioning data from the endoscope, and synchronize them in time; Pattern recognition is performed on the spatial positioning data to identify significant anatomical features in the real-time visual image data; Based on the location of the significant anatomical features and the spatial positioning data, calculate the geometric transformation relationship between the visual coordinate system and the spatial positioning coordinate system of the endoscope; Based on the geometric transformation relationship, the real-time visual image data and the spatial positioning data are fused to display anatomical structure information and surgical tool positions in real time.

[0005] Preferably, the pattern recognition of the spatial positioning data includes: Image quality is evaluated based on the real-time visual image data; Motion features are extracted based on the spatial positioning data; Based on the changes in motion characteristics and image quality, determine the operator's intention to make fine adjustments; When the intention is determined to be spatial calibration, the reliability of the anatomical features in the visual image data is evaluated. If the reliability reaches a preset threshold, the instantaneous recalibration procedure is activated; When the instantaneous recalibration procedure is activated, a new geometric transformation matrix is ​​calculated, and the weights are updated according to the reliability and the purity of the operator. The old and new geometric transformation matrices are then weighted and fused together for a new update.

[0006] Preferably, determining the operator's fine-tuning intention based on the motion characteristics and the image quality changes includes: A preliminary analysis of the motion features and the changes in image quality is performed to identify whether there is a clear intention for purely spatial calibration or field of view optimization. If the preliminary analysis results fail to clearly classify a single intent, a dynamic intent identification cycle is initiated to continuously monitor the operator's fine-tuning actions and changes in image quality. The operator's fine-tuning intentions are dynamically adjusted according to preset intention conversion rules. The confidence threshold for intent judgment is dynamically adjusted based on the duration of the fine-tuning action intent.

[0007] Preferably, when the determination is for spatial calibration, assessing the reliability of anatomical features in the visual image data includes: When the intention is determined to be purely for space calibration, a dynamic evaluation window is initiated. During the dynamic evaluation window, the local sharpness and edge continuity of anatomical features in the visual image data, as well as their positional stability in consecutive image frames, are continuously tracked. The instantaneous reliability score of the anatomical feature is calculated in real time based on the local sharpness, the edge continuity, and the positional stability. Monitor the fluctuation trend of the instantaneous reliability score; If the instantaneous reliability score shows a stable upward trend during the dynamic evaluation window period, and reaches a preset instantaneous high confidence threshold at the end of the dynamic evaluation window period, then the anatomical feature is marked as usable. If the instantaneous reliability score fluctuates drastically or continues to decline during the dynamic evaluation window, the availability rating of the anatomical feature is reduced, and the dynamic evaluation window is extended.

[0008] Preferably, the step of initiating a dynamic evaluation window when the determination is based on a purely spatial calibration intent includes: When the intention is determined to be purely spatial calibration, the duration of the initial dynamic evaluation window is calculated based on the movement speed and angular velocity of the endoscope tip, as well as the initial local sharpness, edge continuity, and positional stability of the anatomical features in the image. During the dynamic evaluation window, the local sharpness, edge continuity, and positional stability of anatomical features in consecutive image frames are continuously monitored, and the instantaneous reliability score is calculated in real time. If the instantaneous reliability score shows a stable upward trend and reaches a high confidence threshold, the window period ends and the anatomical features are marked as available. If the instantaneous reliability score fluctuates drastically or continues to decline, the window period will be extended and a prompt will be triggered.

[0009] Preferably, after calculating the duration of the initial dynamic evaluation window, the method further includes: When the speed or angular velocity of the endoscope changes drastically in a short period of time, or when the local clarity, edge continuity, or positional stability of anatomical features fluctuates significantly between consecutive image frames, the duration of the dynamic evaluation window period is adjusted.

[0010] Preferably, adjusting the duration of the dynamic evaluation window includes: The linear velocity, angular velocity, and rate of change of the linear velocity and angular velocity of the endoscope tip are acquired in real time. Real-time acquisition of local clarity, edge continuity, positional stability, and corresponding rate of change of anatomical features; The expected value of characteristic fluctuations caused by the endoscope's own motion is calculated based on the linear velocity of the endoscope tip, the angular velocity, and the rate of change of the linear velocity and the angular velocity. The actual fluctuation value of the anatomical feature is calculated based on the local clarity of the anatomical feature, the edge continuity, the positional stability, and the corresponding rate of change. Compare the expected value of the characteristic fluctuation with the actual fluctuation value of the anatomical feature; Determine the dominant factor in the fluctuation based on the comparison results; The preset window period is adjusted based on the dominant factors, and the duration of the dynamic evaluation window period is also adjusted.

[0011] Preferably, adjusting the preset window period based on the dominant factor and adjusting the duration of the dynamic evaluation window period includes: When the dominant factor is determined to be a change in the tracheal environment, the severity and duration of the change in the tracheal environment are continuously monitored. If the degree of drastic change in the tracheal environment continues to exceed a preset threshold, and the duration exceeds the preset maximum evaluation window period, then the dynamic evaluation window period is retained. If the severity of the changes in the tracheal environment is alleviated within the maximum assessment window, then the dynamic assessment window period is shortened or maintained. When the dominant factor is determined to be the movement of the endoscope itself, the dynamic evaluation window period is shortened based on the amplitude and frequency of the endoscope's movement.

[0012] Preferably, the continuous monitoring of the severity and duration of changes in the tracheal environment includes: The endoscope's image data is acquired in real time, and the image data is divided into regions to obtain multiple local monitoring areas; Within each local monitoring area, tracheal wall texture features and secretion distribution features were extracted respectively; Time series analysis was performed on the tracheal wall texture features to identify the periodic patterns and amplitudes of physiological peristalsis of the tracheal wall; Time series analysis was performed on the distribution characteristics of the secretions to identify the intensity and frequency of periodic flushing of the secretions; Based on the periodic pattern and amplitude of the physiological peristalsis of the tracheal wall, the expected value of the first image change caused by the peristalsis of the tracheal wall is calculated, and based on the intensity and frequency of the periodic flushing of the secretions, the expected value of the second image change caused by the flushing of the secretions is calculated. The expected value of the first image change and the expected value of the second image change are superimposed to obtain the expected value of the composite environment change; Real-time calculation of the actual change in image quality within the local monitoring area; By comparing the expected value of the composite environmental change with the actual value of the image quality change, the severity of the change in the tracheal environment can be determined. Based on the assessment results, start or update the environmental change duration timer.

[0013] Secondly, the present invention provides an image recognition system for anatomical structures in tracheotomy surgery, comprising: The detection end is used to acquire real-time visual image data and spatial positioning data from the endoscope, and to synchronize them in time. The processing unit is used to perform pattern recognition on the spatial positioning data to identify significant anatomical features in the real-time visual image data; and to calculate the geometric transformation relationship between the visual coordinate system and the spatial positioning coordinate system of the endoscope based on the location of the significant anatomical features and the spatial positioning data. The output end is used to fuse the real-time visual image data and the spatial positioning data according to the geometric transformation relationship, and to display anatomical structure information and surgical tool positions in real time.

[0014] This application discloses a method and system for anatomical structure image recognition in tracheotomy. By acquiring and synchronizing real-time visual image data and spatial positioning data from an endoscope, pattern recognition is performed on the spatial positioning data to identify significant anatomical features in the real-time visual image data. Based on these features and the spatial positioning data, the geometric transformation relationship between the endoscope's visual coordinate system and the spatial positioning coordinate system is calculated. Finally, the data is fused and the anatomical structure information and surgical tool positions are displayed in real time. This method effectively solves the problem in existing technologies where irregular deformation of the endoscope within the narrow and physiologically curved trachea causes subtle changes in the relative geometric relationship between the endoscope's tip camera and the electromagnetic navigation sensor, introducing initial positioning deviations. This leads to slight drifts in the spatial mapping relationship when the system performs the calculation method for establishing the correspondence between spatial position and image pixels during surgery. The technical solution of this application enables real-time correction of the initial transformation matrix deviation introduced by the endoscope's deformation within the trachea, effectively solving the problem of spatial mapping drift, significantly improving the accuracy and safety of tracheotomy, providing doctors with more reliable intraoperative navigation, and reducing surgical risks. Attached Figure Description

[0015] Figure 1 This is a schematic flowchart of an image recognition method for anatomical structures in tracheotomy surgery provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of another method for recognizing anatomical structures in tracheotomy surgery provided by an embodiment of the present invention; Figure 3 This is a schematic diagram of another method for recognizing anatomical structures in tracheotomy surgery provided by an embodiment of the present invention; Figure 4 This is a schematic diagram of an image recognition system for tracheotomy anatomical structures provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Reference Figure 1 The present invention provides a flowchart of an image recognition method for anatomical structures in tracheotomy surgery, comprising the following steps: S1, acquire real-time visual image data and spatial positioning data of the endoscope, and synchronize them in time; S2, perform pattern recognition on the spatial positioning data to identify significant anatomical features in the real-time visual image data; S3, Based on the location of the significant anatomical features and the spatial positioning data, calculate the geometric transformation relationship between the visual coordinate system and the spatial positioning coordinate system of the endoscope; S4. Based on the geometric transformation relationship, the real-time visual image data and the spatial positioning data are fused to display anatomical structure information and surgical tool positions in real time.

[0018] "Endoscope" refers to a medical optical instrument used to observe internal organs through natural body cavities or surgical incisions. In this application, the endoscope is typically equipped with a camera to acquire real-time visual image data and may integrate spatial positioning sensors to provide its position and orientation information in three-dimensional space.

[0019] "Real-time visual image data" refers to the sequence of images continuously captured by an endoscopic camera during surgery, which reflects the anatomical structure inside the trachea.

[0020] "Spatial positioning data" refers to the three-dimensional coordinates and orientation information of the endoscope tip within the patient's body, provided by an electromagnetic navigation system or other positioning system.

[0021] "Time synchronization" refers to ensuring that real-time visual image data and spatial positioning data are aligned on the time axis so that features in the image can be accurately associated with their spatial locations.

[0022] Pattern recognition refers to the use of algorithms to analyze spatial positioning data and identify specific motion patterns or spatial features, such as the fine-tuning, rotation, or translation of an endoscope.

[0023] “Significant anatomical features” refer to structures inside the trachea that have obvious visual characteristics and are easily identifiable, such as the edges of the tracheal cartilage rings, the carina, and the texture of the tracheal wall. These features have high contrast and stability in images.

[0024] The "visual coordinate system" refers to a two-dimensional image coordinate system established with the endoscope camera as the origin.

[0025] A "spatial positioning coordinate system" refers to a three-dimensional coordinate system established by a spatial positioning system, which is usually associated with the patient's anatomical structure or the operating table.

[0026] "Geometric transformation relationship" refers to the mathematical relationship that maps a point in the visual coordinate system to a point in the spatial positioning coordinate system. It is usually represented by a transformation matrix that contains rotation and translation information.

[0027] "Fusion" refers to combining real-time visual image data and spatial positioning data to generate a comprehensive display interface that includes information from both.

[0028] "Anatomical information" refers to the specific location and shape of the internal structures of the trachea determined through image recognition and spatial positioning in fusion display.

[0029] "Surgical instrument position" refers to the real-time position of surgical instruments (such as incision knives, guide wires, etc.) inside the trachea, which is tracked and displayed by a spatial positioning system in the fusion display.

[0030] This application provides a method for image recognition of anatomical structures in tracheotomy surgery, the method comprising the following steps: First, real-time visual image data and spatial positioning data from the endoscope are acquired and synchronized in time. In practice, real-time visual image data is acquired via a miniature camera at the endoscope's tip, which transmits the captured image signal to the image processing unit. Spatial positioning data is acquired via an electromagnetic sensor integrated into the endoscope's tip. This sensor, in conjunction with an external electromagnetic field generator, outputs the real-time position and orientation information of the endoscope tip in three-dimensional space. To ensure consistency between image information and spatial positioning information, these two types of data need to be synchronized in time. For example, a hardware triggering mechanism can be used to ensure that image acquisition and spatial positioning data acquisition occur simultaneously, or software timestamp alignment can be used to post-process the acquired data to eliminate time delays.

[0031] Secondly, pattern recognition is performed on the spatial positioning data to identify significant anatomical features in the real-time visual image data. After acquiring time-synchronized visual image data and spatial positioning data, the spatial positioning data needs to be analyzed to identify the endoscope's movement patterns. For example, by analyzing the displacement and rotation of the endoscope tip within a short period of time, it can be determined whether the operator is performing fine anatomical observation or positioning calibration. When a specific movement pattern is identified (e.g., the endoscope making small-range, high-frequency fine adjustments within a certain area), the system assumes that the operator is attempting to lock onto an important anatomical feature. At this point, the image processing module combines the current visual image data and uses image processing algorithms (such as edge detection, feature point matching, deep learning models, etc.) to identify and lock onto significant anatomical features in the image with high contrast, clear edges, or specific textures, such as the edges of the tracheal cartilage rings, the bifurcation point of the tracheal cartilage, or specific vascular textures.

[0032] Next, based on the location and spatial positioning data of significant anatomical features, the geometric transformation relationship between the endoscope's visual coordinate system and the spatial positioning coordinate system is calculated. Once significant anatomical features in the real-time visual image data are identified, and their pixel coordinates in the image, along with the 3D position and orientation information of the endoscope tip in the spatial positioning coordinate system, the system can calculate the geometric transformation relationship between the visual and spatial positioning coordinate systems. This typically involves solving for a transformation matrix that maps two-dimensional pixels in the image to their actual positions in three-dimensional space. For example, the Perspective-n-Point (PnP) algorithm can be used to estimate the endoscope's pose using the pixel coordinates of multiple known significant anatomical features in the image and their corresponding 3D coordinates in the spatial positioning coordinate system, thus obtaining the transformation matrix from the visual to the spatial positioning coordinate system. In practical applications, iterative optimization algorithms can be used to continuously adjust the transformation matrix to minimize the reprojection error between image feature points and spatial positioning points.

[0033] Finally, based on geometric transformation relationships, real-time visual image data and spatial positioning data are fused to display anatomical structure information and surgical tool positions in real time. After calculating the geometric transformation relationship between the visual coordinate system and the spatial positioning coordinate system, the system can fuse the real-time visual image data acquired by the endoscope with the spatial positioning data. Specifically, the three-dimensional position information of surgical tools (such as incision knives, guidewires, etc.) tracked by the spatial positioning system can be projected onto the real-time visual image through geometric transformation relationships and displayed on the screen in a virtual or augmented reality manner. At the same time, anatomical structure information identified by image recognition algorithms can also be combined with spatial positioning data and displayed as a three-dimensional model or annotation. Thus, doctors can intuitively see the real anatomical structure inside the trachea, the real-time position of surgical tools, and their spatial relationships on a unified display interface, thereby achieving precise surgical navigation.

[0034] This application provides a method for recognizing anatomical structures in tracheotomy. Its working principle is to overcome the positioning deviation problem caused by endoscope deformation in traditional methods by using multimodal data fusion and dynamic calibration mechanism.

[0035] Compared with existing technologies, the advantages of this application are: 1. Higher positioning accuracy: By dynamically calculating geometric transformation relationships, this application can compensate for errors caused by endoscope deformation in real time, which significantly improves the positioning accuracy of tracheal anatomical structures and surgical tools.

[0036] 2. Enhanced environmental adaptability: This application is no longer limited to an ideal calibration environment and can adapt to the complex physiological environment inside the trachea and the actual operational deformation of the endoscope.

[0037] 3. Safer surgical procedures: Precise real-time navigation information can effectively reduce surgical risks, decrease the occurrence of complications, and improve patient safety.

[0038] 4. More intuitive surgical field of view: The integrated display interface provides doctors with more comprehensive and accurate visual information, which helps them make more accurate judgments and operations.

[0039] In some embodiments described above, pattern recognition of spatial positioning data is proposed to identify salient anatomical features in real-time visual image data. However, in actual surgical environments, endoscope movement, changes in the tracheal environment, and operator fine-tuning movements can all affect the accuracy and stability of pattern recognition, leading to insufficient robustness in anatomical feature recognition. This, in turn, affects the accuracy of the geometric transformation relationship between the visual coordinate system and the spatial positioning coordinate system, potentially requiring frequent manual calibration. To address this, this application further proposes a more refined spatial positioning data pattern recognition method. By comprehensively evaluating image quality, motion features, and operator intent, the reliability of anatomical features is dynamically assessed, and an instantaneous recalibration procedure is activated when necessary to achieve adaptive updating of the geometric transformation matrix.

[0040] For details, please refer to Figure 2 S2 includes: S21, evaluate image quality based on real-time visual image data; S22, extract motion features based on spatial positioning data; S23, based on motion characteristics and changes in image quality, determine the operator's intention for fine-tuning actions; S24, when the intention is determined to be spatial calibration, the reliability of anatomical features in the visual image data is evaluated; S25, if the reliability reaches a preset threshold, then the instantaneous recalibration procedure is activated; S26, when the instantaneous recalibration procedure is activated, a new geometric transformation matrix is ​​calculated, and the weights are updated according to the reliability and the purity of the operator's intention, and the old and new geometric transformation matrices are weighted and fused for update.

[0041] Evaluating image quality based on real-time visual image data can be understood as quantitatively analyzing aspects such as image sharpness, contrast, noise level, and illumination uniformity. For example, algorithms such as image gradient, Fourier transform, and information entropy can be used to calculate image sharpness indices, or image histograms can be analyzed to evaluate contrast. The purpose is to provide a foundation for subsequent feature extraction and reliability assessment, ensuring more accurate judgments based on high-quality images.

[0042] Motion feature extraction based on spatial positioning data refers to extracting features reflecting the endoscope's motion state from its spatial positioning data, such as position, attitude, linear velocity, and angular velocity obtained through inertial measurement units or optical tracking systems. For example, it can calculate the endoscope's displacement, rotation angle, and rate of change of velocity over a short period. The purpose is to capture the operator's precise manipulation of the endoscope, providing crucial information for determining their intentions.

[0043] Judging the operator's fine-tuning intentions based on motion characteristics and image quality changes involves comprehensively analyzing the endoscope's motion patterns (such as slow translation and small-amplitude rotation) and image quality trends (such as from blurry to clear, from jittery to stable) to infer whether the operator's current intention is to perform spatial calibration, field of view optimization, or other operations. For example, when the endoscope moves slowly and the image quality gradually improves, it may indicate that the operator is attempting to calibrate the relationship between the endoscope and anatomical structures.

[0044] When spatial calibration is deemed an intention, the reliability of anatomical features in the visual image data is evaluated. This involves quantitatively assessing the stability, sharpness, and consistency of anatomical features (such as tracheal rings and carina) identified in the current image, when the operator explicitly expresses the calibration intention. For example, the tracking stability of feature points, the accuracy of edge detection, and the texture consistency of feature regions can be analyzed. The purpose is to ensure that recalibration is only performed when the anatomical features are sufficiently reliable, avoiding calibration errors caused by unstable features.

[0045] If the reliability reaches a preset threshold, an instantaneous recalibration procedure is activated. This means that when the reliability score of an anatomical feature reaches a pre-set confidence level, the system automatically triggers a rapid, localized recalibration process. This preset threshold can be adjusted based on clinical experience and system performance. Its purpose is to perform calibration at the optimal time to minimize interference with the normal surgical procedure.

[0046] When the instantaneous recalibration procedure is activated, a new geometric transformation matrix is ​​calculated, and the weights are updated based on the reliability and the purity of the operator's intent. A weighted fusion update is then performed on the old and new geometric transformation matrices. The new geometric transformation matrix can be calculated by matching the correspondence between currently reliable anatomical features in the visual coordinate system and the spatial positioning coordinate system. The higher the reliability, the greater the weight of the new matrix; the higher the purity of the operator's intent (i.e., the operator's intent is purely spatial calibration rather than other interference), the greater the weight of the new matrix. The weighted fusion update aims to smooth the transition and avoid instability of the overall system due to instantaneous errors in a single calibration result.

[0047] This application's solution, by introducing image quality assessment of real-time visual image data and extracting motion features from spatial positioning data, can more comprehensively perceive the current surgical environment and the operator's behavior. Through comprehensive analysis of this information, the system can intelligently determine the operator's fine-tuning intentions, thus distinguishing between purely spatial calibration operations and other non-calibration-purpose actions. When a clear spatial calibration intention is identified, the system further evaluates the reliability of anatomical features in the visual image data, ensuring that subsequent calibration operations are only performed when the anatomical features are clear, stable, and reliable. If the reliability of the anatomical features reaches a preset threshold, an instantaneous recalibration procedure is activated to calculate a new geometric transformation matrix. During this process, the weights of the new geometric transformation matrix are dynamically adjusted according to the reliability of the anatomical features and the purity of the operator's intention, and are weighted and fused with the old geometric transformation matrix for updating. This achieves adaptive, smooth, and robust calibration of geometric transformation relationships, effectively avoiding problems such as inaccurate pattern recognition and geometric transformation relationship drift caused by environmental changes or operational uncertainties.

[0048] Through the above technical solution, this application can significantly improve the accuracy and stability of anatomical structure image recognition during tracheotomy. Compared with the basic solution, this application introduces image quality assessment, motion feature extraction, and operator intent judgment, enabling the system to intelligently identify the calibration timing and avoid invalid or erroneous calibration when image quality is poor or operator intent is unclear. Furthermore, dynamically evaluating the reliability of anatomical features and updating the geometric transformation matrix based on a weighted fusion of the reliability and intent purity ensures the robustness and adaptability of the calibration process. This effectively suppresses recognition drift caused by factors such as minor endoscopic vibrations, tracheal wall peristalsis, or interference from secretions, thereby providing doctors with more accurate and stable anatomical structure information and surgical tool position display, greatly improving the safety and efficiency of the surgery.

[0049] As a specific implementation, it is assumed that during a tracheotomy, real-time visual image data from the endoscope is continuously acquired, while its spatial positioning data (e.g., provided by an electromagnetic tracking system) is also collected synchronously. The system first performs image quality assessment on the real-time visual image data, such as calculating an image sharpness score, and extracts motion features from the spatial positioning data, such as calculating the linear and angular velocities of the endoscope tip. When the system detects that the endoscope is making small-range translations or rotations at a low speed, and the image sharpness score shows a stable upward trend, the system determines that the operator currently intends spatial calibration. At this point, the system further evaluates the reliability of significant anatomical features in the image, such as the tracheal rings, for example, by tracking the stability of these feature points and the continuity of edge detection. If the reliability score reaches a preset threshold of 0.8, the system activates an instantaneous recalibration procedure. In this procedure, a new geometric transformation matrix is ​​calculated based on the currently reliable anatomical features. Simultaneously, based on the reliability of the current anatomical features (e.g., 0.9) and the purity of the operator's intention (e.g., judged to be a pure calibration intention, with a purity of 0.95), the system updates the weights, for example, the new matrix weight is 0.7 and the old matrix weight is 0.3. Then, the new and old geometric transformation matrices are weighted and fused to update, thereby obtaining a more accurate and stable geometric transformation relationship between the visual coordinate system and the spatial positioning coordinate system.

[0050] In some of the embodiments described above in this application, the operator's fine-tuning intention is determined based on motion characteristics and changes in image quality. However, in actual operation, the operator's fine-tuning intention may not always be singular and clear. Sometimes there may be a mixture of multiple intentions, or the intention may change in a short period of time. This may lead to an inaccurate or untimely judgment of the operator's true intention, thereby affecting the efficiency and accuracy of subsequent spatial calibration or field of view optimization.

[0051] In this regard, this application further proposes S23, which includes: S231, a preliminary analysis is performed on the motion features and the changes in image quality to identify whether there is an obvious intention for pure spatial calibration or field of view optimization. S232, when the preliminary analysis results fail to clearly classify a single intent, the dynamic intent recognition cycle is initiated to continuously monitor the operator's fine-tuning actions and changes in image quality; S233, dynamically adjust the operator's fine-tuning intentions according to preset intention conversion rules; S234, dynamically adjust the confidence threshold for intent judgment based on the duration of the operator's fine-tuning action.

[0052] Specifically, a preliminary analysis of motion characteristics and image quality changes aims to quickly determine whether the operator has a clear, singular intention by instantly evaluating motion characteristics such as the endoscope's trajectory, speed, and acceleration, as well as image quality indicators such as image sharpness, contrast, and edge sharpness in real-time visual image data. For example, when the endoscope moves slowly and steadily with continuously improving image quality, it may indicate a purely spatial calibration intention; while when the endoscope moves rapidly to cover a new area and image quality changes drastically in a short period of time, it may indicate a field of view optimization intention.

[0053] When the initial analysis results fail to clearly classify a single intent, it means that the operator's main intent cannot be determined through a preliminary and rapid judgment. In this case, the system will activate a dynamic intent recognition cycle to continuously monitor the operator's fine-tuning actions and changes in image quality over a longer time scale and with finer granularity, thereby collecting more information for a more accurate judgment.

[0054] In practical applications, dynamically adjusting the operator's fine-tuning intent based on preset intent conversion rules means that the system maintains a set of rules that define how intent transitions from one state (e.g., "potential calibration") to another (e.g., "confirm calibration" or "convert to field of view optimization") under different monitoring data (e.g., motion patterns, image change trends, duration). These rules can be based on expert experience, machine learning models, or predefined logic. Furthermore, dynamically adjusting the confidence threshold for intent judgment based on the duration of the operator's fine-tuning action aims to ensure that a longer duration indicates a clearer and more stable intent, allowing for a lower confidence threshold for faster intent confirmation. Conversely, shorter and less stable action durations require a higher confidence threshold to avoid misjudgments.

[0055] This application's solution effectively addresses the ambiguity and uncertainty inherent in traditional methods when judging the operator's fine-tuning intentions by introducing mechanisms such as preliminary analysis, a dynamic intent identification cycle, intent transition rules, and confidence threshold adjustment based on duration. First, preliminary analysis quickly identifies clear, single intents, improving response speed. Second, when the intent is ambiguous, the activation of the dynamic intent identification cycle ensures the system continuously collects more contextual information, avoiding erroneous judgments based on instantaneous data. Furthermore, pre-defined intent transition rules allow the system to flexibly adjust its judgment of the operator's intent based on constantly changing monitoring data, enhancing the adaptability of the judgment. Finally, by dynamically adjusting the confidence threshold based on the duration of the fine-tuning action, this application's solution more intelligently balances judgment speed and accuracy, confirming stable intents with long durations more quickly, while remaining cautious with short-lived or unstable intents, thus significantly improving the accuracy and robustness of recognizing the operator's true intent.

[0056] Through the above technical solution, this application can more accurately and robustly determine the operator's intention for fine-tuning movements during tracheotomy. Compared to methods that rely solely on instantaneous data for judgment, this application effectively addresses the complexity and dynamism of the operator's intentions by introducing a dynamic intention recognition cycle and intention transition rules, avoiding misjudgments caused by ambiguous or changing intentions. Furthermore, by dynamically adjusting the confidence threshold based on the duration of the fine-tuning movement, the system can respond flexibly according to the stability of the intention while ensuring judgment accuracy. This improves the efficiency and reliability of subsequent spatial calibration or visual field optimization, ultimately contributing to enhanced surgical safety and precision.

[0057] In some preferred embodiments, a specific example is illustrated below. Suppose that during a tracheotomy, the endoscopic operator performs a series of fine-tuning movements. The system first performs a preliminary analysis of the endoscope's motion characteristics (such as speed and direction changes) and image quality changes (such as sharpness and contrast) in the real-time visual image data. If the preliminary analysis shows that the endoscope is translating in a certain direction at a very slow speed, and the edge sharpness of a certain anatomical feature in the image is continuously and steadily improving, the system may initially determine it as a pure spatial calibration intention. However, if the preliminary analysis results show that the endoscope has performed multiple small translations and rotations in a short period of time, while the image quality is alternately improving in different areas, and cannot be clearly classified as a single intention, the system will initiate a dynamic intention recognition cycle. During this cycle, the system continuously monitors the operator's fine-tuning movement patterns (e.g., whether a specific sequence of movements is repeated) and image quality change trends (e.g., whether the sharpness of a specific anatomical region is continuously improving). For example, if the system observes that the operator repeatedly makes small left-right movements over a continuous 5 seconds, and the texture of the tracheal wall in the center of the field of vision becomes clearer after each movement, the system will dynamically adjust the operator's intention from "unclear" to "field of vision optimization intention" according to preset intention conversion rules. At the same time, because this fine-tuning action lasts for a relatively long time, the system will appropriately lower the confidence threshold required to determine the "field of vision optimization intention," thereby confirming the intention more quickly and triggering the corresponding field of vision optimization processing, such as adjusting image display parameters or suggesting that the operator further adjust the endoscope position to obtain the best field of vision.

[0058] In some embodiments described above, this application proposes assessing the reliability of anatomical features in visual image data when spatial calibration is deemed an intention. However, in actual surgical environments, real-time visual image data from a tracheoscopic endoscope may be affected by various dynamic factors, such as slight endoscope movement, physiological peristalsis of the tracheal wall, periodic flushing of secretions, or transient changes in illumination. These factors may cause fluctuations in the local clarity, edge continuity, or positional stability of anatomical features over a short period. If only static or transient reliability assessments are performed, they may not accurately reflect the true stability of the anatomical features, thus introducing errors, affecting the accuracy of subsequent transient recalibration procedures, and consequently reducing the precision of surgical navigation.

[0059] In response, this application further proposes a method for assessing the reliability of anatomical features in visual image data when the intention is determined to be spatial calibration, specifically including: When the intent is determined to be purely spatial calibration, a dynamic evaluation window is initiated. "Pure spatial calibration intent" refers to a fine-tuning action by the operator that is explicitly and unambiguously intended for spatial calibration, rather than for other purposes, such as simply optimizing the field of view. This intent can be determined based on the specific pattern and duration of the operator's fine-tuning action, as well as the specific response to changes in image quality. Once this pure intent is identified, the system initiates a "dynamic evaluation window," a preset or dynamically adjusted time period used to continuously monitor and evaluate the reliability of anatomical features.

[0060] During the dynamic evaluation window, the local sharpness and edge continuity of anatomical features in the visual image data, as well as their positional stability across consecutive image frames, are continuously tracked. Specifically, "local sharpness" can be quantified by calculating metrics such as the image gradient magnitude, Laplacian operator variance, or frequency domain energy of the anatomical feature region to reflect the sharpness of that region. "Edge continuity" can be evaluated by analyzing the coherence and smoothness of the anatomical feature boundaries, for example, by using the Canny edge detection algorithm to extract edges and then fitting or performing connectivity analysis on the edge chains. "Positional stability" is measured by tracking the center point or key points of the anatomical feature across consecutive image frames and calculating their displacement or jitter amplitude over time, for example, using optical flow or feature point matching algorithms.

[0061] Based on the local sharpness, edge continuity, and positional stability, the instantaneous reliability score of the anatomical feature is calculated in real time. This "instantaneous reliability score" is a comprehensive quantitative indicator that can be calculated by fusing the above three parameters through weighted averaging, fuzzy logic reasoning, or machine learning models to reflect the overall reliability of the anatomical feature at the current moment.

[0062] The system monitors the fluctuation trend of the instantaneous reliability score. It continuously records and analyzes the instantaneous reliability score. The dynamic assessment window period is used to determine whether the changes show an upward, downward, or drastic fluctuation trend.

[0063] If the instantaneous reliability score shows a stable upward trend during the dynamic evaluation window and reaches a preset instantaneous high confidence threshold at the end of the dynamic evaluation window, then the anatomical feature is marked as usable. This means that the image quality and stability of the anatomical feature continuously improve over a period of time and reach a sufficiently high level, indicating that it is suitable for subsequent instantaneous recalibration.

[0064] If the instantaneous reliability score fluctuates drastically or continues to decline within the dynamic evaluation window, the availability rating of the anatomical feature is reduced, and the dynamic evaluation window is extended. When the reliability score of an anatomical feature is unstable or continues to deteriorate, it indicates that the feature is currently unsuitable for calibration. The system will reduce its availability rating and may extend the evaluation window to allow more time to observe whether it can recover stability or to confirm that it is indeed unusable.

[0065] This application's solution effectively addresses the limitations of traditional static assessment by introducing a dynamic evaluation window and continuously tracking multiple image quality metrics for anatomical features. Specifically, when the operator expresses a purely spatial calibration intent, the system no longer relies solely on the image quality at a single instant but initiates a dynamic evaluation process. During this process, by continuously monitoring the local sharpness, edge continuity, and positional stability of anatomical features and calculating their instantaneous reliability scores in real time, the system can comprehensively and dynamically capture the true state of anatomical features in complex tracheal environments. Furthermore, by monitoring the fluctuation trend of the instantaneous reliability score, the system can distinguish between transient disturbances and continuous stability improvements. Only when the score shows a stable upward trend and reaches a preset high confidence threshold is the anatomical feature marked as usable, ensuring that the anatomical features used for recalibration have high reliability and stability. Conversely, if the score fluctuates drastically or continuously declines, the availability rating is lowered and the evaluation window is extended to avoid using unstable features for calibration, thereby improving the accuracy and robustness of the instantaneous recalibration procedure.

[0066] Through the above technical solution, this application overcomes the problem of inaccurate anatomical feature reliability assessment in dynamic surgical environments. This solution introduces a dynamic assessment window and continuously tracks the local sharpness, edge continuity, and positional stability of anatomical features. Combined with the judgment of fluctuation trends in instantaneous reliability scores, it ensures that anatomical features are only used for instantaneous recalibration when both image quality and stability meet high standards. This significantly improves the accuracy and robustness of the instantaneous recalibration procedure, effectively avoiding geometric transformation relationship calculation errors caused by the use of unstable or low-quality anatomical features. Consequently, it provides more accurate and reliable anatomical structure information and surgical tool position display for tracheotomy, greatly enhancing the safety and efficiency of the surgery.

[0067] In some preferred embodiments, a specific example is illustrated below. Suppose that during a tracheotomy, the operator makes fine adjustments using an endoscope, and the system determines through pattern recognition that the operator intends purely spatial calibration. At this point, the system immediately initiates a dynamic evaluation window, e.g., lasting 5 seconds. During this window, the system continuously tracks a significant anatomical feature in the endoscopic field of view, such as the edge of the tracheal ring. At the beginning of the window, due to slight endoscope movement, the local clarity of the tracheal ring may be slightly low, the edge continuity may be slightly interrupted, and the positional stability may fluctuate slightly, resulting in a moderate instantaneous reliability score. However, as the operator stabilizes the endoscope, the system continuously monitors that the local clarity of the tracheal ring gradually improves, the edge becomes more continuous, and its position in consecutive image frames tends to stabilize. Therefore, the instantaneous reliability score shows a steady upward trend. When the 5-second window ends, if the instantaneous reliability score has reached a preset instantaneous high-confidence threshold (e.g., 0.9), the tracheal ring anatomical feature is marked as usable and can be used for subsequent instantaneous recalibration. Conversely, if, during the assessment window, a patient's sudden cough causes violent peristalsis of the tracheal wall, resulting in severe blurring and displacement of the tracheal ring image, and the instantaneous reliability score fluctuates drastically or continues to decline, the system will lower the availability rating of that anatomical feature and may extend the assessment window to wait for the environment to stabilize or to find other reliable features, thereby avoiding inaccurate recalibration under unstable conditions.

[0068] In some of the above embodiments, when the intention is determined to be purely spatial calibration, a dynamic evaluation window is initiated to continuously track the reliability of anatomical features. However, in practical applications, if the duration of the dynamic evaluation window is fixed or its initial duration is not optimized according to real-time operating conditions and image quality, it may lead to low evaluation efficiency or inaccurate evaluation results. For example, when the endoscope is stable and the anatomical features are clear, an excessively long evaluation window will unnecessarily prolong the waiting time; while when the endoscope is unstable or the anatomical features are of poor quality, an excessively short evaluation window may not be able to collect enough data to accurately assess reliability, and may even miss critical calibration opportunities.

[0069] In response, this application further proposes a method for adaptive management of the dynamic assessment window period, which aims to dynamically adjust the duration of the assessment window period based on real-time operating conditions and anatomical feature quality, thereby improving the efficiency and accuracy of the assessment.

[0070] When the intention is determined to be purely spatial calibration, a dynamic evaluation window is initiated, including: When the intention is determined to be purely spatial calibration, the duration of the initial dynamic evaluation window is calculated based on the movement speed and angular velocity of the endoscope tip, as well as the initial local sharpness, edge continuity, and positional stability of the anatomical features in the image. During the dynamic evaluation window, the local sharpness, edge continuity, and positional stability of anatomical features in consecutive image frames are continuously monitored, and the instantaneous reliability score is calculated in real time. If the instantaneous reliability score shows a stable upward trend and reaches a high confidence threshold, the window period ends and the anatomical features are marked as available. If the instantaneous reliability score fluctuates drastically or continues to decline, the window period will be extended and a prompt will be triggered.

[0071] Specifically, when the system determines the intent to perform purely spatial calibration, it calculates the duration of the initial dynamic evaluation window based on multiple real-time parameters, including the endoscope tip's movement speed and angular velocity, as well as the initial local sharpness, edge continuity, and positional stability of anatomical features in the image. This means the window length is no longer a preset fixed value but is intelligently adjusted according to the current surgical environment and operational status. For example, when the endoscope moves quickly or has a large angular velocity, or when the initial local sharpness, edge continuity, or positional stability of the anatomical features is low, the system may calculate a relatively long initial window to ensure sufficient time to collect stable data and assess feature reliability. Conversely, if the endoscope movement is smooth and the initial quality of the anatomical features is high, a shorter initial window can be calculated to improve efficiency.

[0072] During the dynamic evaluation window, the system continuously monitors the local sharpness, edge continuity, and positional stability of anatomical features across consecutive image frames, and calculates instantaneous reliability scores in real time. This process is consistent with the continuous tracking and real-time calculation of instantaneous reliability scores in the aforementioned method, aiming to dynamically reflect the quality changes of anatomical features during the evaluation period.

[0073] In practical applications, if the instantaneous reliability score shows a stable upward trend and reaches a high confidence threshold, the system will end the window period early and mark the anatomical feature as available. This mechanism allows the system to complete the evaluation immediately when the reliability of the anatomical feature rapidly reaches a high level, avoiding unnecessary waiting and thus accelerating the subsequent instantaneous recalibration procedure.

[0074] Furthermore, if the instantaneous reliability score fluctuates drastically or continues to decline, the system will extend the window period and trigger a prompt. This adaptive extension mechanism is designed to address adverse situations that may occur during the assessment process, such as unexpected endoscope jitter, obstructed view, or temporary blurring of anatomical features. By extending the window period, the system can attempt to collect data over a longer period, hoping that the anatomical features can recover and stabilize. Simultaneously, the trigger prompt can promptly inform the operator that the reliability assessment of the current anatomical features is encountering problems, prompting the operator to adjust the endoscope position or take other measures to improve image quality, thereby increasing the success rate of the assessment and the final calibration accuracy.

[0075] This application's solution effectively addresses the efficiency and accuracy issues that can arise from traditional fixed or non-adaptive window periods by introducing a mechanism for calculating the initial window period based on real-time parameters and dynamically adjusting the window period according to reliability score trends. Specifically, by comprehensively considering the initial quality of the endoscope's motion and anatomical features to determine the initial window period, it ensures that the assessment process matches the current surgical environment from the outset, avoiding the uncertainty caused by blindly setting the window period. It is precisely this intelligent initial setting that allows the assessment process to be initiated more rationally.

[0076] Furthermore, by continuously monitoring the trend of instantaneous reliability scores during the evaluation process and dynamically ending or extending the window period accordingly, this approach can flexibly respond to various dynamic changes that may occur during surgery. When reliability rapidly increases and reaches a high confidence level, ending the window period early can significantly shorten waiting time and improve the efficiency of the surgical procedure. Conversely, when reliability fluctuates or declines, extending the window period and triggering prompts provides the system with additional opportunities to overcome temporary interference and guides the operator to optimize operations, thereby ensuring that the anatomical feature data ultimately used for geometric transformation relationship calculations has higher reliability and stability. This two-way adaptive adjustment mechanism makes the evaluation process both efficient and robust.

[0077] Through the above technical solution, this application can significantly improve the efficiency and accuracy of anatomical structure image recognition in tracheotomy. Specifically, by calculating the initial dynamic assessment window period based on the endoscope's motion state and the initial quality of anatomical features, the problem of under- or over-assessment that may arise from a fixed window period is avoided, making the assessment process more accurate and efficient. Furthermore, the dynamic adjustment mechanism based on the instantaneous reliability score trend enables the system to intelligently complete the assessment quickly when reliability meets the standard, or extend the assessment and provide operator prompts when reliability is poor, thus effectively coping with complex and changing surgical environments. This adaptive management approach not only optimizes resource utilization and reduces unnecessary waiting time, but also further improves the reliability of the identified anatomical features through timely feedback and operator guidance, ultimately providing a solid foundation for more accurate geometric transformation relationship calculations and surgical navigation, thereby improving the safety and success rate of the surgery.

[0078] As a specific implementation, suppose that during a tracheotomy, an endoscope is rapidly inserted and initially positioned. At this point, the system determines it is a purely spatial calibration attempt. Due to the relatively high speed and angular velocity of the endoscope tip, and the unstable initial local sharpness, edge continuity, and positional stability of anatomical features in the image, the system calculates a relatively long initial dynamic evaluation window, for example, set to 5 seconds. During the subsequent evaluation, the operator gradually stabilizes the endoscope, causing the local sharpness, edge continuity, and positional stability of the anatomical features to gradually improve, and the instantaneous reliability score also steadily increases. When the window reaches 3 seconds, the instantaneous reliability score has stably reached the preset high confidence threshold. At this point, the system immediately ends the dynamic evaluation window and marks the anatomical features as usable, thus entering the instantaneous recalibration procedure 2 seconds earlier than originally planned, significantly improving efficiency.

[0079] In another example, suppose the endoscope is stable, but the periodic flushing of tracheal secretions causes intermittent decreases in the local clarity of anatomical features, resulting in drastic fluctuations in the instantaneous reliability score. In this case, the system will determine that the reliability has decreased based on the fluctuation trend and automatically extend the dynamic assessment window, for example, from the initial 3 seconds to 6 seconds. Simultaneously, the system will trigger a visual or auditory cue informing the operator that the reliability assessment of the current anatomical features is being interfered with, suggesting that the operator try adjusting the endoscope position or wait for the flushing of secretions to end. In this way, the system can more robustly handle assessment tasks in complex environments, ensuring that reliable anatomical feature data can be obtained as much as possible even under adverse conditions.

[0080] In some embodiments described above, a dynamic assessment window is initiated when the intention is determined to be purely spatial calibration. The duration of the initial dynamic assessment window is calculated based on the endoscope tip's movement speed, angular velocity, and the initial local sharpness, edge continuity, and positional stability of the anatomical features in the image. However, in actual surgery, the endoscope's motion and the intratracheal environment may change drastically in a short period, causing significant fluctuations in the image quality of the anatomical features. In such cases, the window duration calculated solely based on initial conditions may not adequately adapt to these dynamic changes, thus affecting the accurate assessment of the reliability of the anatomical features.

[0081] In this regard, this application further proposes that after calculating the duration of the initial dynamic evaluation window period, when the endoscope's motion speed or angular velocity changes drastically in a short period of time, or when the local clarity, edge continuity, or positional stability of anatomical features fluctuates significantly between consecutive image frames, the duration of the dynamic evaluation window period shall be adjusted.

[0082] Specifically, "when the speed or angular velocity of the endoscope changes drastically in a short period of time" refers to a significant increase or decrease in the linear velocity, angular velocity, or both of the endoscope tip within a very short time interval, such as when the operator performs rapid translation, rotation, or tilting operations. Such drastic changes may cause rapid movement or shaking of the image field, making the representation of anatomical features unstable in the image.

[0083] "Significant fluctuations in the local sharpness, edge continuity, and positional stability of anatomical features across consecutive image frames" refers to a significant decrease or instability in the image quality of a specific anatomical feature when it is tracked within a dynamic evaluation window. For example, local sharpness may suddenly decrease, edges may become blurred or broken, or the feature may exhibit irregular and rapid drifting in position across consecutive image frames, rather than smooth motion.

[0084] "Adjusting the duration of the dynamic assessment window" refers to recalculating or correcting the remaining duration of the currently ongoing dynamic assessment window based on drastic changes or significant fluctuations in the quality of detected endoscopic motion or anatomical feature images. This adjustment may involve extending the window to wait for conditions to stabilize, shortening the window to avoid ineffective assessments under unstable conditions, or determining a more suitable assessment duration based on new dynamic conditions.

[0085] The solution proposed in this application overcomes the aforementioned limitations by continuously monitoring the motion state of the endoscope and the image quality of anatomical features after the initial dynamic assessment window period is calculated. When a drastic change in the endoscope's motion speed or angular velocity is detected, or when significant fluctuations occur in the local clarity, edge continuity, or positional stability of anatomical features, the system can promptly identify a significant change in the current assessment environment. It is precisely this real-time monitoring and judgment mechanism that allows the system to flexibly adjust the preset duration of the dynamic assessment window period based on the actual dynamic situation. For example, if a drastic change causes a temporary decrease in image quality, the system can extend the window period to allow the image to stabilize, ensuring sufficient time to acquire reliable assessment data; if the change is persistent and irreversible, the system can shorten the window period to avoid wasting computational resources on invalid data. This dynamic adjustment mechanism ensures that the assessment of the reliability of anatomical features is always based on the most suitable observation period in complex and ever-changing surgical environments.

[0086] Through the above technical solution, this application can significantly improve the adaptability and robustness of the tracheotomy anatomical structure image recognition method in dynamic environments. This solution makes the dynamic assessment window period no longer fixed, but intelligently adjusted according to the real-time movement of the endoscope and changes in the image quality of anatomical features. This effectively avoids the problem of mismatched assessment windows due to sudden operations or environmental changes, thus ensuring the accuracy and timeliness of the reliability assessment of anatomical features. Therefore, during surgery, even in the face of rapid movement or image quality fluctuations, the system can more accurately determine the availability of anatomical features, providing more reliable input for subsequent instantaneous recalibration procedures, thereby improving the accuracy and safety of overall surgical navigation.

[0087] In some preferred embodiments, assuming that during a tracheotomy, the system has calculated and initiated a 5-second dynamic assessment window based on the initial motion parameters of the endoscope tip and the initial image quality of the anatomical features. At the second second of the window, due to an unforeseen situation, the operator rapidly adjusts the endoscope's position, causing a sharp increase in the endoscope's linear and angular velocities within 0.5 seconds. Simultaneously, the target anatomical feature (e.g., the tracheal rings) appears significantly blurred and has broken edges in the image, and its position drifts rapidly. At this point, the solution of this application detects in real-time the drastic changes in the endoscope's motion speed and angular velocity, as well as significant fluctuations in the local clarity, edge continuity, and positional stability of the anatomical features. Based on these real-time changes, the system immediately adjusts the duration of the remaining dynamic assessment window. For example, the system might determine that the current environment is unsuitable for reliability assessment and extend the window by 3 seconds to allow the operator to stabilize the endoscope and restore image quality; or, if it determines that such fluctuations are persistent and cannot be recovered in a short time, the system might shorten the window and trigger a prompt, suggesting that the operator re-stabilize the field of view. Through this dynamic adjustment, the system can avoid ineffective evaluations under unstable conditions and ensure that an accurate judgment on the reliability of anatomical features is made only when conditions are suitable.

[0088] This application further proposes the above-mentioned method for adjusting the duration of the dynamic assessment window, which includes: The linear velocity, angular velocity, and rate of change of the linear velocity and angular velocity of the endoscope tip are acquired in real time. Real-time acquisition of local clarity, edge continuity, positional stability, and corresponding rate of change of anatomical features; The expected value of characteristic fluctuations caused by the endoscope's own motion is calculated based on the linear velocity of the endoscope tip, the angular velocity, and the rate of change of the linear velocity and the angular velocity. The actual fluctuation value of the anatomical feature is calculated based on the local clarity of the anatomical feature, the edge continuity, the positional stability, and the corresponding rate of change. Compare the expected value of the characteristic fluctuation with the actual fluctuation value of the anatomical feature; Determine the dominant factor in the fluctuation based on the comparison results; The preset window period is adjusted based on the dominant factors, and the duration of the dynamic evaluation window period is also adjusted.

[0089] Specifically, real-time acquisition of the endoscope tip's linear velocity, angular velocity, and the rates of change of these velocities refers to calculating the endoscope's translational velocity (linear velocity), rotational velocity (angular velocity), and their respective rates of change over time in three-dimensional space using the inertial measurement unit (IMU) integrated into the endoscope or from continuous spatial positioning data. These parameters reflect the endoscope's real-time motion state and trends. Real-time acquisition of the local sharpness, edge continuity, positional stability, and corresponding rates of change of anatomical features refers to image processing and analysis of significant anatomical features in real-time endoscopic visual image data. Local sharpness can be quantified using methods such as image gradient, contrast, or frequency domain analysis; edge continuity can be evaluated using edge detection algorithms and connectivity analysis; and positional stability is measured through feature point tracking or region matching in consecutive image frames. The rates of change of these indicators reflect the dynamic changes in the visual quality and spatial position of anatomical features.

[0090] The calculation of the expected value of feature fluctuations caused by the endoscope's own motion, based on the linear velocity, angular velocity, and rates of change of the endoscope tip, refers to predicting the degree of blurring, displacement, or deformation of anatomical features in the image under the current endoscopic motion state, based on the endoscope's kinematic model and camera parameters. For example, high-speed motion or violent rotation may lead to image blurring or rapid changes in feature positions; these expected values ​​can serve as a benchmark for the impact of the endoscope's own motion on image quality. In practical applications, the calculation of the actual fluctuation value of anatomical features, based on the local sharpness, edge continuity, positional stability, and corresponding rates of change, refers to comprehensively quantifying the changes in various visual indicators of anatomical features observed in real-time images to obtain a numerical value reflecting the overall degree of fluctuation. This value directly reflects the instability of the anatomical features in the image.

[0091] Furthermore, by comparing the expected fluctuation value of the feature with the actual fluctuation value of the anatomical feature, the dominant factor of the fluctuation can be determined. If the actual fluctuation value is close to the expected fluctuation value, it indicates that the endoscope's own movement is the main cause of the fluctuation. Conversely, if the actual fluctuation value is significantly greater than the expected fluctuation value, it suggests the presence of other factors, such as changes in the tracheal environment (e.g., secretion flushing, physiological peristalsis of the tracheal wall, or patient respiratory movements), which have additional impacts on the image quality and positional stability of the anatomical feature. Therefore, based on the determined dominant factor, the preset dynamic assessment window period can be adjusted accordingly to ensure that the window period duration more accurately adapts to the current surgical environment and operational status.

[0092] This application's solution optimizes the adjustment strategy for the dynamic assessment window period by real-time monitoring of the endoscope's motion and changes in the image quality of anatomical features, and further distinguishing the dominant factors causing these fluctuations. Specifically, by calculating the expected value of feature fluctuations caused by the endoscope's own motion and comparing it with the actual fluctuation values ​​of anatomical features extracted from the image data, the system can intelligently determine whether the current fluctuation is mainly caused by endoscopic operation or dominated by changes in the tracheal environment. This differentiation mechanism makes the adjustment of the dynamic assessment window period no longer a single response, but can be targeted and optimized according to the nature of the fluctuation source. For example, when the endoscope's own motion is the dominant factor, the window period can be quickly adjusted to adapt to the operator's intention; while when changes in the tracheal environment are the dominant factor, a longer window period may be needed to smooth out environmental noise, or a prompt may be triggered to wait for the environment to stabilize, thereby ensuring that interference from non-operational factors can be eliminated or effectively compensated when assessing the reliability of anatomical features, improving the accuracy and robustness of the assessment.

[0093] Through the above technical solution, this application enables more precise and intelligent adjustment of the duration of the dynamic assessment window period. Compared to solutions that adjust solely based on fluctuations, this application identifies the dominant factors of these fluctuations, avoiding inappropriate window period adjustments due to misjudgment of the source of fluctuations. For example, when the tracheal environment changes drastically, the system can identify this dominant factor and adopt a more appropriate window period strategy, rather than simply shortening or lengthening the window period, thereby effectively reducing the interference of environmental factors on the reliability assessment of anatomical features. Therefore, this application significantly improves the accuracy, stability, and adaptability of anatomical feature reliability assessment in the complex and ever-changing tracheal environment, providing more reliable input for subsequent instantaneous recalibration procedures, and thus improving the overall performance and safety of anatomical structure image recognition during tracheotomy.

[0094] This application further proposes adjusting the preset window period based on the aforementioned dominant factors, and adjusting the duration of the dynamic evaluation window period, specifically including: When the dominant factor is determined to be a change in the tracheal environment, the severity and duration of the change in the tracheal environment are continuously monitored. If the degree of drastic change in the tracheal environment continues to exceed a preset threshold, and the duration exceeds the preset maximum evaluation window period, then the dynamic evaluation window period is retained. If the severity of the changes in the tracheal environment is alleviated within the maximum assessment window, then the dynamic assessment window period is shortened or maintained. When the dominant factor is determined to be the movement of the endoscope itself, the dynamic evaluation window period is shortened based on the amplitude and frequency of the endoscope's own movement.

[0095] Specifically, "changes in the tracheal environment" can be understood as image changes caused by physiological peristalsis, secretion flushing, and respiratory movements within the trachea. Continuously monitoring the severity and duration of these tracheal environment changes involves using image processing algorithms, such as texture analysis and optical flow, to quantify the amplitude of tracheal wall movement, the flow velocity and range of secretions, and to record the duration of these changes. The preset threshold is a critical value set during system design based on clinical experience or experimental data to distinguish between mild and severe fluctuations.

[0096] The maximum assessment window period refers to an upper limit set for the dynamic assessment window period to prevent the window period from being extended indefinitely in extreme cases. When the severity of changes in the tracheal environment is consistently higher than a preset threshold and the duration exceeds the preset maximum assessment window period, it indicates that the tracheal environment is continuously unstable. In this case, the dynamic assessment window period is retained to give the system sufficient time to wait for the environment to stabilize, or to find a possible moment of stability for assessment during the continuous instability, avoiding premature abandonment of assessment due to the continuously deteriorating environment.

[0097] If the severity of changes in the intratracheal environment improves within the maximum assessment window, for example, gradually stabilizing from severe fluctuations, then shortening or maintaining the dynamic assessment window is necessary to respond promptly to environmental improvements, accelerate the assessment process, and avoid unnecessary waiting. Shortening the window period can be understood as proportionally reducing the remaining window time based on the degree of improvement, while maintaining the window period means keeping the current window length for continued observation when environmental improvement is not significant but there is still hope.

[0098] When the dominant factor is determined to be endoscope movement itself, the dynamic assessment window period is shortened based on the amplitude and frequency of this movement. The amplitude of endoscope movement refers to the displacement distance of the endoscope tip in space, and the frequency refers to the number of movements or the rate of change in speed per unit time. Since endoscope movement is usually controllable by the operator, and the resulting image fluctuations are highly predictable, when it is determined that the fluctuations are mainly caused by endoscope movement, it can be assumed that the operator is capable of stabilizing the image by adjusting their operation. Shortening the window period in this case aims to encourage the system to complete the assessment more quickly or prompt the operator to adjust their operation, thereby improving efficiency.

[0099] Through the aforementioned technical solution, this application can finely and adaptively adjust the duration of the dynamic assessment window period based on the specific dominant factors causing fluctuations in anatomical features. This avoids the problems of low assessment efficiency or insufficient accuracy that may result from a single, fixed window period adjustment strategy. Specifically, when the intratracheal environment is continuously unstable, the system can maintain sufficient assessment time, ensuring no potential recalibration opportunities are missed; while when the environment improves or endoscopic movement is controllable, the system can respond quickly, shortening unnecessary waiting times, thereby significantly improving the robustness, real-time performance, and overall efficiency of the instantaneous recalibration procedure. This differentiated adjustment mechanism enables the system to assess the reliability of anatomical features more intelligently and effectively in complex and changing surgical environments, providing a more stable and accurate foundation for subsequent geometric transformation relationship calculations and surgical navigation.

[0100] In some preferred embodiments, a specific example is given below. Suppose that during a tracheotomy, the system analyzes the linear and angular velocities of the endoscope tip, as well as the local sharpness, edge continuity, positional stability, and rate of change of anatomical features in the image, determining that the fluctuations in current anatomical features are mainly caused by changes in the tracheal intra-arterial environment. At this point, the system will initiate continuous monitoring of the severity and duration of these changes. For example, if the system detects a sustained large amplitude of physiological peristalsis of the tracheal wall and frequent flushing by secretions, resulting in severe fluctuations in image quality, and this severity consistently exceeds a preset threshold (e.g., image sharpness decreases by more than 10%), and the duration has exceeded the preset maximum evaluation window period (e.g., 30 seconds), then the system will choose to retain the current dynamic evaluation window period and continue observation in order to capture reliable anatomical features when the environment briefly stabilizes.

[0101] On the other hand, if the system detects a reduction in the severity of changes in the tracheal environment during monitoring—for example, a decrease in tracheal wall peristalsis, a reduction in the frequency of secretion flushing, and a stabilization in image quality fluctuations—and this reduction occurs within the maximum assessment window, the system will dynamically shorten or maintain the dynamic assessment window period based on the degree of reduction. For instance, if the degree of fluctuation decreases from severe to slight, the system may shorten the remaining window period by 50% to accelerate the assessment process.

[0102] The continuous monitoring of the severity and duration of changes in the tracheal environment includes the following steps: The endoscope's image data is acquired in real time, and the image data is divided into regions to obtain multiple local monitoring areas; Within each local monitoring area, tracheal wall texture features and secretion distribution features are extracted; time series analysis is performed on the tracheal wall texture features to identify the periodic patterns and amplitudes of physiological peristalsis of the tracheal wall; Time series analysis was performed on the distribution characteristics of the secretions to identify the intensity and frequency of periodic flushing of the secretions; Based on the periodic pattern and amplitude of the physiological peristalsis of the tracheal wall, the expected value of the first image change caused by the peristalsis of the tracheal wall is calculated, and based on the intensity and frequency of the periodic flushing of the secretions, the expected value of the second image change caused by the flushing of the secretions is calculated. The expected change values ​​of the first image and the second image are superimposed to obtain the expected change value of the composite environment; the actual change value of image quality within the local monitoring area is calculated in real time. By comparing the expected value of the composite environmental change with the actual value of the image quality change, the severity of the change in the tracheal environment can be determined. Based on the judgment result, start or update the environmental change duration timer.

[0103] Real-time acquisition of endoscopic image data refers to the continuous acquisition of video streams or image frames captured by the endoscope during the surgical procedure. The image data is then divided into regions to obtain multiple local monitoring areas. The purpose of this is to break down the entire field of view into smaller, independently analyzable areas, allowing for more precise capture of local environmental changes, such as specific parts of the tracheal wall or areas where secretions accumulate.

[0104] Within each local monitoring area, tracheal wall texture features and secretion distribution features are extracted. Tracheal wall texture features can be understood as visual information such as the fine structure, color, and gloss of the tracheal inner wall surface, and their extraction can be achieved through image processing algorithms, such as Gabor filters, Local Binary Pattern (LBP) or deep learning feature extractors. Secretion distribution features refer to information such as the morphology, location, density, and dynamic changes of secretions within the trachea, and their extraction can be achieved through methods such as color segmentation, morphological manipulation, or fluid motion analysis.

[0105] Time-series analysis of the tracheal wall texture features identifies the periodic patterns and amplitudes of physiological peristalsis of the tracheal wall. The aim is to quantify the impact of the natural physiological movements of the tracheal wall on the image. For example, Fourier transform or wavelet analysis can be used to identify the periodic contraction and relaxation patterns of the tracheal wall from consecutive image frames, and the intensity of these movements can be calculated. Time-series analysis of the secretion distribution features identifies the intensity and frequency of periodic secretion flushing. The aim is to quantify the interference caused by the flow and flushing of secretions within the trachea on the image. For example, optical flow or background subtraction methods can be used to track the trajectory and velocity of secretions, thereby assessing the intensity and frequency of flushing.

[0106] Based on the periodic pattern and amplitude of the physiological peristalsis of the tracheal wall, a first expected value of image change caused by tracheal wall peristalsis is calculated, and a second expected value of image change caused by the periodic flushing of secretions is calculated based on the intensity and frequency of the flushing. These expected values ​​represent the natural fluctuations in image quality that may occur under normal physiological or environmental conditions. The first and second expected values ​​of image change are superimposed to obtain a composite environmental change expected value, the purpose of which is to comprehensively consider the combined effects of multiple physiological and environmental factors within the trachea on the image.

[0107] The actual change in image quality within the monitored local area is calculated in real time to obtain the true fluctuation of image quality at the current moment. This actual change value can be calculated based on indicators such as local sharpness, edge continuity, contrast, or noise level. Subsequently, the expected value of the composite environmental change is compared with the actual change value of the image quality to determine the severity of the change in the intratracheal environment. If the actual change value significantly exceeds the expected value, it indicates an abnormal or severe environmental change. Based on the determination result, an environmental change duration timer is started or updated to record and track the duration of the environmental change, providing a temporal basis for subsequent window period adjustments.

[0108] The proposed method, through detailed local monitoring and feature extraction of endoscopic image data, combined with time-series analysis, can accurately distinguish between normal image fluctuations caused by physiological peristalsis of the tracheal wall and the flushing effect of secretions, and drastic fluctuations caused by other factors (such as endoscope movement or abnormal environmental changes). By calculating and comparing the expected value of composite environmental changes with the actual value of image quality changes, the severity of changes in the tracheal environment can be objectively determined, thus providing a reliable basis for subsequent dynamic assessment and adjustment of the window period.

[0109] The above technical solutions enable refined and dynamic monitoring of changes in the tracheal environment, effectively distinguishing between physiological fluctuations and non-physiological disturbances, thereby more accurately assessing the reliability of anatomical features. This helps avoid misjudging the usability of anatomical features due to normal fluctuations in the tracheal environment, improves the triggering accuracy of the instantaneous recalibration procedure and the robustness of the geometric transformation matrix update, and thus enhances the overall accuracy and stability of anatomical structure image recognition during tracheotomy.

[0110] Reference Figure 4 This invention provides an image recognition system for anatomical structures in tracheotomy surgery, comprising: The detection end is used to acquire real-time visual image data and spatial positioning data from the endoscope, and to synchronize them in time. The processing unit is used to perform pattern recognition on the spatial positioning data to identify significant anatomical features in the real-time visual image data; and to calculate the geometric transformation relationship between the visual coordinate system and the spatial positioning coordinate system of the endoscope based on the location of the significant anatomical features and the spatial positioning data. The output end is used to fuse the real-time visual image data and the spatial positioning data according to the geometric transformation relationship, and to display anatomical structure information and surgical tool positions in real time.

[0111] It should be noted that the tracheotomy anatomical structure image recognition system provided in this embodiment of the invention is used to execute all the process steps of the tracheotomy anatomical structure image recognition method of the above embodiment. The working principle and beneficial effects of the two are one-to-one, so they will not be described again.

[0112] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for image recognition of anatomical structures in tracheotomy surgery, characterized in that, The method includes: Acquire real-time visual image data and spatial positioning data from the endoscope, and synchronize them in time; Pattern recognition is performed on the spatial positioning data to identify significant anatomical features in the real-time visual image data; Based on the location of the significant anatomical features and the spatial positioning data, calculate the geometric transformation relationship between the visual coordinate system and the spatial positioning coordinate system of the endoscope; Based on the geometric transformation relationship, the real-time visual image data and the spatial positioning data are fused to display anatomical structure information and surgical tool positions in real time.

2. The method for image recognition of anatomical structures in tracheotomy surgery according to claim 1, characterized in that, The pattern recognition of the spatial positioning data includes: Image quality is evaluated based on the real-time visual image data; Motion features are extracted based on the spatial positioning data; Based on the changes in motion characteristics and image quality, determine the operator's intention to make fine adjustments; When the intention is determined to be spatial calibration, the reliability of the anatomical features in the visual image data is evaluated. If the reliability reaches a preset threshold, the instantaneous recalibration procedure is activated; When the instantaneous recalibration procedure is activated, a new geometric transformation matrix is ​​calculated, and the weights are updated according to the reliability and the purity of the operator. The old and new geometric transformation matrices are then weighted and fused together for a new update.

3. The method for image recognition of anatomical structures in tracheotomy surgery according to claim 2, characterized in that, The step of determining the operator's fine-tuning intention based on the motion characteristics and the image quality changes includes: A preliminary analysis of the motion features and the changes in image quality is performed to identify whether there is a clear intention for purely spatial calibration or field of view optimization. If the preliminary analysis results fail to clearly classify a single intent, a dynamic intent identification cycle is initiated to continuously monitor the operator's fine-tuning actions and changes in image quality. The operator's fine-tuning intentions are dynamically adjusted according to preset intention conversion rules. The confidence threshold for intent judgment is dynamically adjusted based on the duration of the fine-tuning action intent.

4. The method for image recognition of anatomical structures in tracheotomy surgery according to claim 2, characterized in that, When the determination is for spatial calibration intent, the reliability of the anatomical features in the visual image data is evaluated, including: When the intention is determined to be purely for space calibration, a dynamic evaluation window is initiated. During the dynamic evaluation window, the local sharpness and edge continuity of anatomical features in the visual image data, as well as their positional stability in consecutive image frames, are continuously tracked. The instantaneous reliability score of the anatomical feature is calculated in real time based on the local sharpness, the edge continuity, and the positional stability. Monitor the fluctuation trend of the instantaneous reliability score; If the instantaneous reliability score shows a stable upward trend during the dynamic evaluation window period, and reaches a preset instantaneous high confidence threshold at the end of the dynamic evaluation window period, then the anatomical feature is marked as usable. If the instantaneous reliability score fluctuates drastically or continues to decline during the dynamic evaluation window, the availability rating of the anatomical feature is reduced, and the dynamic evaluation window is extended.

5. The method for image recognition of anatomical structures in tracheotomy surgery according to claim 4, characterized in that, When the determination is made to be a purely spatial calibration intention, a dynamic evaluation window is initiated, including: When the intention is determined to be purely spatial calibration, the duration of the initial dynamic evaluation window is calculated based on the movement speed and angular velocity of the endoscope tip, as well as the initial local sharpness, edge continuity, and positional stability of the anatomical features in the image. During the dynamic evaluation window, the local sharpness, edge continuity, and positional stability of anatomical features in consecutive image frames are continuously monitored, and the instantaneous reliability score is calculated in real time. If the instantaneous reliability score shows a stable upward trend and reaches a high confidence threshold, the window period ends and the anatomical features are marked as available. If the instantaneous reliability score fluctuates drastically or continues to decline, the window period will be extended and a prompt will be triggered.

6. The method for image recognition of anatomical structures in tracheotomy surgery according to claim 5, characterized in that, After calculating the duration of the initial dynamic evaluation window, the method further includes: When the speed or angular velocity of the endoscope changes drastically in a short period of time, or when the local clarity, edge continuity, or positional stability of anatomical features fluctuates significantly between consecutive image frames, the duration of the dynamic evaluation window period is adjusted.

7. The method for image recognition of anatomical structures in tracheotomy surgery according to claim 6, characterized in that, Adjusting the duration of the dynamic evaluation window includes: The linear velocity, angular velocity, and rate of change of the linear velocity and angular velocity of the endoscope tip are acquired in real time. Real-time acquisition of local clarity, edge continuity, positional stability, and corresponding rate of change of anatomical features; The expected value of characteristic fluctuations caused by the endoscope's own motion is calculated based on the linear velocity of the endoscope tip, the angular velocity, and the rate of change of the linear velocity and the angular velocity. The actual fluctuation value of the anatomical feature is calculated based on the local clarity of the anatomical feature, the edge continuity, the positional stability, and the corresponding rate of change. Compare the expected value of the characteristic fluctuation with the actual fluctuation value of the anatomical feature; Determine the dominant factor in the fluctuation based on the comparison results; The preset window period is adjusted based on the dominant factors, and the duration of the dynamic evaluation window period is also adjusted.

8. The method for image recognition of anatomical structures in tracheotomy surgery according to claim 7, characterized in that, The step of adjusting the preset window period based on the dominant factors and adjusting the duration of the dynamic evaluation window period includes: When the dominant factor is determined to be a change in the tracheal environment, the severity and duration of the change in the tracheal environment are continuously monitored. If the degree of drastic change in the tracheal environment continues to exceed a preset threshold, and the duration exceeds the preset maximum evaluation window period, then the dynamic evaluation window period is retained. If the severity of the changes in the tracheal environment is alleviated within the maximum assessment window, then the dynamic assessment window period is shortened or maintained. When the dominant factor is determined to be the movement of the endoscope itself, the dynamic evaluation window period is shortened based on the amplitude and frequency of the endoscope's movement.

9. The method for image recognition of anatomical structures in tracheotomy surgery according to claim 7, characterized in that, The continuous monitoring of the severity and duration of changes in the tracheal environment includes: The endoscope's image data is acquired in real time, and the image data is divided into regions to obtain multiple local monitoring areas; Within each local monitoring area, tracheal wall texture features and secretion distribution features were extracted respectively; Time series analysis was performed on the tracheal wall texture features to identify the periodic patterns and amplitudes of physiological peristalsis of the tracheal wall; Time series analysis was performed on the distribution characteristics of the secretions to identify the intensity and frequency of periodic flushing of the secretions; Based on the periodic pattern and amplitude of the physiological peristalsis of the tracheal wall, the expected value of the first image change caused by the peristalsis of the tracheal wall is calculated, and based on the intensity and frequency of the periodic flushing of the secretions, the expected value of the second image change caused by the flushing of the secretions is calculated. The expected value of the first image change and the expected value of the second image change are superimposed to obtain the expected value of the composite environment change; Real-time calculation of the actual change in image quality within the local monitoring area; By comparing the expected value of the composite environmental change with the actual value of the image quality change, the severity of the change in the tracheal environment can be determined. Based on the assessment results, start or update the environmental change duration timer.

10. A system for recognizing anatomical structures during tracheotomy, characterized in that, The system includes: The detection end is used to acquire real-time visual image data and spatial positioning data from the endoscope, and to synchronize them in time. The processing unit is used to perform pattern recognition on the spatial positioning data to identify significant anatomical features in the real-time visual image data; and to calculate the geometric transformation relationship between the visual coordinate system and the spatial positioning coordinate system of the endoscope based on the location of the significant anatomical features and the spatial positioning data. The output end is used to fuse the real-time visual image data and the spatial positioning data according to the geometric transformation relationship, and to display anatomical structure information and surgical tool positions in real time.

Images