Visual guide type tracheal intubation endoscope system and control method

Abstract

The invention relates to the technical field of medical instruments and intelligent control, and discloses a visual guiding type trachea cannula endoscope system and a control method. Firstly, a pixel coordinate system is established for a visual field image of the intubation mirror, pixel-physical calibration is completed, and coordinate normalization is achieved; then gray normalization and multi-direction gradient calculation are carried out on the image, edges and tiny structures are accurately extracted, obstacles are segmented through a self-adaptive threshold value, connected domain analysis is carried out, and a centroid is extracted; then combining with the tip radius of the intubation lens to construct an attraction-repulsion potential field, iteratively generating a discrete track under the guidance of gradient descent of the synthetic potential field, converting the discrete track into a steering engine control instruction, and automatically prompting when the system encounters a local minimum value or iteration exceeds a limit to avoid algorithm failure; finally, the path and prompt information are overlaid into the image in real time, and visual, accurate and steady intubation navigation is provided for an operator.

Description

Technical Field

[0001] This invention relates to the field of medical devices and intelligent control technology, specifically to a visually guided endotracheal intubation endoscope system and control method. Background Technology

[0002] Endotracheal intubation is a crucial procedure in clinical resuscitation, anesthesia, and critical care, requiring the accurate and safe delivery of the intubation scope to the glottis and into the trachea within a limited timeframe. Traditional endotracheal intubation methods rely primarily on the operator's visual experience and manual manipulation, with common auxiliary tools including a conventional laryngoscope and a fiberoptic bronchoscope.

[0003] During intubation, the operator needs to observe the image under the endoscope on the display screen and manually control the direction and speed of the intubation endoscope based on subjective judgment, experience, and repeated fine adjustments. The lack of intelligent path planning and obstacle avoidance mechanisms can easily increase the difficulty of the operation due to complex airway structures, secretions, tumors, or malformations. While existing video intubation endoscopes can achieve real-time image display, they are mainly based on "passive display" and cannot achieve automatic detection and intelligent obstacle avoidance of airway obstacles. Faced with narrow airways or complex airway lesions, existing equipment struggles to provide operators with quantitative and visual path planning and navigation suggestions, easily leading to risks such as prolonged operation time, intubation failure, or tissue damage. With the development of medical image processing and intelligent control theory, some studies have attempted to use image-based assisted localization and navigation methods, such as image segmentation, edge detection, and deep learning recognition. However, these methods often rely on large-scale sample training or model generalization capabilities and mostly remain at the auxiliary analysis stage, lacking a path guidance mechanism that can provide real-time feedback, dynamic adjustment, and clear physical meaning. Especially in high-risk intubation scenarios with special anatomical structures, traditional static image algorithms and simple rules of thumb are difficult to meet the dual requirements of real-time performance and safety.

[0004] Therefore, this study aims to propose a visually guided endotracheal intubation system and its control method. Through pixel-to-physical mapping calibration, image grayscale normalization, and gradient analysis, the system automatically identifies and segments obstacle structures within the airway. Subsequently, based on the spatial distribution of obstacles and the target location, combined with the physical characteristics of the intubation scope, a custom attraction-repulsion potential field is constructed. Using a potential field gradient descent strategy, a collision-free path from the current starting point to the glottic target is planned in real time, progressively generating discrete navigation points and converting them into servo control commands to achieve intelligent dynamic obstacle avoidance and propulsion of the intubation scope. The system also overlays the planned path and prompts onto the real-time field of view, providing the operator with intuitive and dynamic navigation assistance. Summary of the Invention

[0005] This invention provides a visually guided endotracheal intubation system and control method, which helps to solve the problems mentioned in the background art.

[0006] This invention provides the following technical solution: a visually guided endotracheal intubation endoscope control method, comprising:

[0007] The field of view images acquired by the intubation endoscope are initialized, a two-dimensional pixel coordinate system with the upper left corner of the image as the origin is established, and the conversion relationship between pixels and actual physical length is determined through physical calibration.

[0008] The image's pixels are normalized to grayscale, and the gradient magnitude of each pixel is calculated based on the normalized grayscale values.

[0009] By setting a global threshold, pixels above the threshold are classified as obstacle regions. Connectivity analysis is then performed on the obstacle regions to extract the centroid coordinates of each connected region.

[0010] Based on the physical radius of the tip of the cannula, the attractive potential field function of the target point and the repulsive potential field function of each obstacle are established in the image space, and the composite potential field is obtained by function synthesis.

[0011] The gradient of the synthetic potential field at any point is calculated, and the direction of motion of the cannula tip is determined accordingly.

[0012] Based on the direction of motion and the set step size, discrete trajectory points are generated iteratively, and path termination is determined according to the set conditions.

[0013] The displacement vector and steering angle of each segment of the generated discrete trajectory are calculated, and the corresponding actuator commands are generated based on the calculation results.

[0014] The generated path and related guidance information are overlaid on the real-time acquired images to assist operators in performing intubation procedures.

[0015] Optionally, the initialization of the field-of-view image acquired by the intubation endoscope, establishing a two-dimensional pixel coordinate system with the upper left corner of the image as the origin, and determining the conversion relationship between pixels and actual physical length through physical calibration, specifically includes:

[0016] Obtain the width and height of the current field of view image, denoted as W and H respectively;

[0017] Taking the top left corner of the image as the origin O pix Establish a pixel coordinate system with the horizontal direction to the right as the positive x-axis and the vertical direction downward as the positive y-axis, with the unit being pixels and the pixel coordinates being (x, y).

[0018] Calculate the length of the image diagonal

[0019] The operator marks the pixel coordinates (x, y) at both ends of a known physical length ruler in the view. A ,y A ) and (xB ,y B ), and calculate the length N of the scale in the pixel coordinate system. ref :

[0020]

[0021] The actual physical length of the reference scale is denoted as L. ref ;

[0022] Calculate the pixel-to-physical length conversion ratio.

[0023] The surgeon selects the pixel coordinates of the endoscope tip and records them as follows:

[0024] The surgeon selects the pixel coordinates of the center of the glottis, and records them as follows:

[0025] Obtain the positions p0 and p1 of the intubation endoscope's starting and target points in the normalized coordinate system, respectively. g :

[0026]

[0027] The lower limit protection constant is set to δ = 10. -6 .

[0028] Optionally, the step of performing grayscale normalization on each pixel of the image and calculating the gradient magnitude of each pixel based on the normalized grayscale values ​​specifically includes:

[0029] Obtain the original grayscale value at pixel coordinates (x, y), denoted as I(x, y);

[0030] Get the minimum grayscale value of all pixels I min =min 1≤x≤W,1≤y≤H I(x,y);

[0031] Get the maximum grayscale value of all pixels I max =max 1≤x≤W,1≤y≤H I(x,y);

[0032] Calculate the normalized gray value G(x,y) of pixel (x,y):

[0033]

[0034] Construct the gradient components in the x-direction respectively. gradient components in the y-direction

[0035]

[0036] Calculate the gradient magnitude M(x,y) of pixel (x,y):

[0037]

[0038] Optionally, the step of setting a global threshold to classify pixels above the threshold as obstacle regions, performing connected component analysis on the obstacle regions, and extracting the centroid coordinates of each connected component specifically includes:

[0039] Calculate the mean gradient magnitude of the entire image.

[0040] Calculate the standard deviation of gradient magnitude across the entire map.

[0041] Set a global threshold T = μ for obstacle pixel detection. M +σ M ;

[0042] Construct a label matrix Label(x,y) = 0 and a region number q = 0;

[0043] For each pixel (x,y), if M(x,y)≥T and Label(x,y)=0, execute steps S301 to S303:

[0044] S301, Update q←q+1;

[0045] S302. Initialize queue Q and enqueue (x,y), set Label(x,y) = q;

[0046] S302. When Q is not empty, dequeue (x′,y′), check the 4-neighborhood {(x′±1,y′),(x′,y′±1)}, for each neighboring point (u,v): if 1≤u≤W, 1≤v≤H, M(u,v)≥T and Label(u,v)=0, then set Label(u,v)=q and enqueue (u,v);

[0047] Calculate the centroid of the j-th connected component in the pixel coordinate system.

[0048] Where j is the connected component number, j = {1,...,q}; Let x be the centroid coordinate of the j-th connected component; The y-coordinate of the centroid of the j-th connected component; |C j | represents the number of pixels in the j-th connected component;

[0049] Obtain the dimensionless coordinates of the j-th connected component.

[0050] Optionally, based on the physical radius of the cannula tip, the attraction potential field function of the target point and the repulsion potential field function of each obstacle are established in the image space, and the composite potential field is obtained by function synthesis, specifically including:

[0051] Obtain the physical radius of the endotracheal intubation scope tip, denoted as R. tip ;

[0052] Calculate the number of pixels R corresponding to the tip radius respectively. pix And the normalized value R of the tip radius:

[0053]

[0054] Attractive potential field function U att (p) causes the trajectory to converge toward the target point: Where, ||·|| is the Euclidean norm; p = (p x ,p y ) represents the current dimensionless coordinates; p g The coordinates of the target point are dimensionless.

[0055] Calculate the distance d between the current point and the centroid of the obstacle. j =max(‖po j ‖,δ); where o j Let be the dimensionless coordinates of the centroid of the j-th obstacle;

[0056] Construct the repulsive potential function U of the j-th obstacle rep,j (p), to prevent the path from getting too close to obstacles:

[0057]

[0058] Constructing the synthetic potential field

[0059] Optionally, the step of calculating the gradient of the synthetic potential field at any point and determining the direction of motion of the cannula tip accordingly specifically includes:

[0060] Construct the gradient of the attractive potential at p

[0061] Construct the gradient of the j-th barrier repulsion potential field

[0062] Calculate the gradient of the composite potential field

[0063] Obtain the magnitude of the gradient of the synthetic potential field

[0064] Construct the trajectory and plan the direction at point p.

[0065] Optionally, the step of iteratively generating discrete trajectory points based on the direction of motion and a set step size, and determining path termination according to set conditions, specifically includes:

[0066] Set the maximum step size s for each iteration. max =R, maximum number of iterations

[0067] Set the initial dimensionless coordinates to p0, the current iteration step k = 0, and the branch identifier string branch = NULL;

[0068] Repeat steps S601 to S606 until the exit condition is triggered:

[0069] S601. Calculate the gradient magnitude g at the k-th step. k =g(p k If g k If p ≤ δ, set branch = gradient_zero and exit; where p k The normalized coordinates of the k-th discrete trajectory point;

[0070] S602. Calculate the distance d between the k-th step and the target point. k =‖p k -p g ||; if d k ≤s max , let p k+1 =p g branch=reached_gold, exit;

[0071] S603, If k≥K max Set branch = max_iter, then exit;

[0072] S604. Otherwise, calculate p. k+1 =p k +s max d(p k );

[0073] S605, Boundary Constraints: p k+1,x p k+1,y Limited to [0,1];

[0074] S606, Let k ← k+1;

[0075] S607, Branch Response:

[0076] If branch = gradient_zero, it indicates that the potential field gradient is approaching zero, which may be trapped in the extreme region, requiring manual intervention or reselection of the target.

[0077] If branch = max_iter, the message indicates that the number of iterations is too high and the path has not reached the destination. Please check for obstacles or adjust the parameters.

[0078] If branch = reached_gold, it indicates that the target has been reached and the path generation is complete;

[0079] Output trajectory {p0,p1,...,p K}; where K is the step number of the last step.

[0080] Optionally, the step of calculating the displacement vector and steering angle for each segment of the generated discrete trajectory, and generating corresponding actuator commands based on the calculation results, specifically includes:

[0081] Construct the displacement vector Δp ​​of the k-th segment of the trajectory k =p k+1 -p k =(Δx) k ,Δy k ); where Δx k ,Δy k These are the x and y coordinate increments at the k-th step, respectively;

[0082] Calculate the turning angle θ of the k-th segment of the trajectory. k =atan2(Δy k ,Δx k ); where atan2(a,b) is the arctangent function, which returns the angle between the vector (b,a) and the x-axis;

[0083] Each θ k The commands are packaged into servo steering protocol instructions and sent to the intubation endoscope execution system in real time.

[0084] Optionally, the generated path and related guidance information are overlaid and displayed in the real-time acquired images to assist the operator in performing the intubation operation, specifically including:

[0085] Inverse normalization of coordinates: in, Here are the pixel coordinates corresponding to the k-th trajectory point; round(·) is the rounding function; p kx With p ky For the k-th trajectory point, the x and y coordinate components are normalized.

[0086] In each frame of the image, connect all the lines sequentially. Path display is generated;

[0087] Highlight the next target point that is about to be reached using a dot;

[0088] At the same time, the branch hint information obtained in step S607 is displayed on the visualization interface.

[0089] A system for implementing the aforementioned visualization-guided endotracheal intubation endoscope control method includes:

[0090] Image acquisition unit: used to acquire real-time field-of-view images;

[0091] Coordinate system calibration unit: used to establish the pixel coordinate system and calibrate the pixel with the physical scale;

[0092] Image processing unit: used to perform grayscale normalization, gradient calculation, obstacle segmentation, and centroid extraction;

[0093] Potential field construction and planning unit: used to construct the synthetic potential field and solve for the motion direction vector and trajectory generation;

[0094] Control command generation unit: used to calculate steering angle and generate servo control protocol commands;

[0095] Visualization display unit: Used to overlay path and prompt information onto the real-time field-of-view image.

[0096] The present invention has the following beneficial effects:

[0097] 1. By automatically mapping real-time pixel coordinates to a physical scale, the endoscopic workspace is seamlessly normalized, establishing a standard space adaptable to different patients and equipment resolutions. Through rapid operator selection, the system can adapt to any imaging conditions and endoscopic dimensions without the need for specialized calibration materials. This not only ensures the uniformity of physical dimensions in subsequent path algorithms but also lowers the system deployment threshold, enhancing the versatility and clinical applicability of the approach. Compared to existing technologies that typically rely on specific equipment parameters, resolutions, or preset models, this approach improves flexibility and reliability.

[0098] 2. An adaptive grayscale normalization algorithm with local extremum protection and boundary-safe gradient algorithm is employed to ensure efficient and robust highlighting of real obstacle structure boundaries under different lighting, noise, or surface reflection conditions. Compared to traditional methods that rely solely on global thresholding or simple filtering, this full-range, distributionally adaptive gradient strategy enhances the robustness of image structure segmentation, reduces false positives and false negatives, and lays the foundation for subsequent accurate obstacle avoidance.

[0099] 3. By employing a global gradient statistics adaptive threshold and a breadth-first connected component growth algorithm, the obstacle structure is automatically separated into multiple physically independent regions, and the geometric center of each obstacle is efficiently extracted. Compared to existing solutions that require manual parameter adjustment or reliance on deep learning models for segmentation, this solution requires no training samples or parameter presets and can directly adapt to obstacle recognition in any new scenario. This effectively improves the dynamic recognition and tracking capability of the intubation endoscope for complex airway obstacles and reduces blind spots.

[0100] 4. The attraction-repulsion potential field design based on the physical dimensions of the endoscope and obstacle distribution overcomes the limitations of existing static or purely empirical paths. The attraction potential field directly drives the path to converge toward the glottis target, while the repulsion potential field is strictly limited to be effective within the actual safe radius of the endoscope, ensuring dynamic obstacle avoidance and prevention of mechanical damage even in extremely narrow, deformed, or adhered airways. Compared with traditional empirical methods or image-based shortest path planning methods, this scheme can respond in real time to changes in airway structure and instrument dimensions, and has clear physical meaning and interpretability.

[0101] 5. A normalized potential field gradient descent trajectory search method is proposed, and branch protection is implemented to address gradient vanishing and extreme points. Compared to classic greedy algorithms or uncontrollable AI models, this approach ensures that each step proceeds along the optimal direction and can proactively identify and alert to risks such as local extrema, improving the reliability and controllability of path planning. It also effectively solves the risk of "path dead ends" or entering dead zones in clinical practice, enhancing intubation safety.

[0102] 6. By employing physical constraints of step size and iteration upper limit, and automatically prompting for trajectory termination branches, this approach ensures that the path converges within a safe and finite number of steps or triggers an alarm in a timely manner, avoiding "dead loops" or blind advancement. Unlike existing automated intubation systems without termination protection or purely manual navigation, this solution can automatically adapt to various complex clinical structures, promptly report problems, and improve the robustness and usability of the navigation system.

[0103] 7. The turning angles after path point differentiation are directly converted into standard servo control commands, ensuring real-time and accurate correspondence between the intubation scope's movement and path planning. Unlike traditional single-step manual remote control or navigation methods that only provide auxiliary prompts, this solution achieves closed-loop automatic guidance of the intubation scope, reducing human error and unnecessary operational steps.

[0104] 8. Path points and branching indicators are overlaid on the endoscopic view in real time, providing the operator with dynamic and intuitive intubation path and status feedback. Unlike existing video laryngoscopes or intubation endoscopes that only display images, this solution reduces the operator's cognitive burden and improves operational efficiency and safety. Dynamic path and abnormal indications allow the operator to monitor the intubation progress and risks at any time, achieving efficient collaboration between intelligence and human intervention. Attached Figure Description

[0105] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0106] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0107] Example, refer to Figure 1 A visually guided endotracheal intubation endoscope control method, comprising:

[0108] The field of view images acquired by the intubation endoscope are initialized, a two-dimensional pixel coordinate system with the upper left corner of the image as the origin is established, and the conversion relationship between pixels and actual physical length is determined through physical calibration.

[0109] The image's pixels are normalized to grayscale, and the gradient magnitude of each pixel is calculated based on the normalized grayscale values.

[0110] By setting a global threshold, pixels above the threshold are classified as obstacle regions. Connectivity analysis is then performed on the obstacle regions to extract the centroid coordinates of each connected region.

[0111] Based on the physical radius of the tip of the cannula, the attractive potential field function of the target point and the repulsive potential field function of each obstacle are established in the image space, and the composite potential field is obtained by function synthesis.

[0112] The gradient of the synthetic potential field at any point is calculated, and the direction of motion of the cannula tip is determined accordingly.

[0113] Based on the direction of motion and the set step size, discrete trajectory points are generated iteratively, and path termination is determined according to the set conditions.

[0114] The displacement vector and steering angle of each segment of the generated discrete trajectory are calculated, and the corresponding actuator commands are generated based on the calculation results.

[0115] The generated path and related guidance information are overlaid on the real-time acquired images to assist operators in performing intubation procedures.

[0116] By performing a series of steps, including initialization and coordinate calibration of the intubation endoscope image, grayscale normalization and gradient calculation, obstacle segmentation and connected component centroid extraction, potential field function construction and gradient solving, discrete trajectory generation and servo command conversion, and real-time visualization overlay, this method solves the problems of high blindness, insufficient spatial positioning accuracy, and difficulty in ensuring safety during traditional endotracheal intubation navigation, which rely on manual judgment. Specifically, firstly, by establishing a pixel coordinate system and combining it with a physical scale calibrated by the surgeon, accurate mapping of the image space to the real surgical space is ensured, avoiding scale deviations caused by differences in equipment resolution or installation position. Next, full-image adaptive grayscale normalization and multi-directional gradient amplitude calculation are used to ensure stable identification of edges and small structures even under complex lighting and mucus interference, laying the foundation for accurate obstacle segmentation. On this basis, the centroid of each obstacle region is extracted through connected component analysis, and then combined with the radius of the intubation endoscope tip to construct an attractive and repulsive potential field, which not only ensures active approach to the target point but also dynamically avoids surrounding obstacles, forming a smooth and continuous synthetic potential field. The gradient of the synthetic potential field at the current position is then calculated to determine the optimal direction of motion in real time. Discrete trajectory points are generated iteratively, and prompts are issued when a minimum value is encountered or the path cannot be continued, preventing the algorithm from looping endlessly. Finally, the generated navigation path and prompts are superimposed on the real-time image, providing the operator with intuitive and easy-to-understand path guidance. This method, while balancing real-time response, navigation accuracy, and obstacle avoidance safety, achieves a closed-loop process from image preprocessing to mechanical execution and visualization, providing an efficient, robust, and controllable intelligent assistance solution for complex airway intubation procedures.

[0117] The initialization of the field-of-view image acquired by the intubation endoscope, establishing a two-dimensional pixel coordinate system with the upper left corner of the image as the origin, and determining the conversion relationship between pixels and actual physical length through physical calibration, specifically includes:

[0118] Obtain the width and height of the current field of view image, denoted as W and H respectively; obtain the width and height of the original image output by the current camera or display system to establish a basic data environment for subsequent coordinate normalization and path visualization;

[0119] Taking the top left corner of the image as the origin O pix Establish a pixel coordinate system with the positive x-axis pointing horizontally to the right and the positive y-axis pointing vertically downwards, with the unit being pixels and the pixel coordinates being (x, y); define a pixel-level coordinate reference so that each pixel position in the image has a unique and standardized (x, y) identifier.

[0120] Calculate the length of the image diagonal Calculate the diagonal length of the current view pixel coordinate system and use it as the denominator for normalizing all pixel positions, so that coordinate transformations under different resolutions and devices have uniformity and compatibility.

[0121] The operator marks the pixel coordinates (x, y) at both ends of a known physical length ruler in the view. A ,y A ) and (x B ,y B ), and calculate the length N of the scale in the pixel coordinate system. ref :

[0122] Measure the pixel distance of the user-selected physical ruler on the image to establish a unique and traceable quantitative mapping relationship between pixel length and actual physical length;

[0123] The actual physical length of the reference scale is denoted as L. ref ;

[0124] Calculate the pixel-to-physical length conversion ratio. Calculate the scaling factor from pixel to physical distance based on the actual length of the scale and the pixel distance to provide a physical scale basis for the subsequent dimensional calibration and spatial planning of intubation instruments;

[0125] The surgeon selects the pixel coordinates of the endoscope tip and records them as follows:

[0126] The surgeon selects the pixel coordinates of the center of the glottis, and records them as follows:

[0127] Record the positions of the tip of the intubation endoscope and the center of the target (glottis) in the pixel coordinate system as the starting and ending points of path planning, which directly affects path solving and servo control;

[0128] Obtain the positions p0 and p1 of the intubation endoscope's starting and target points in the normalized coordinate system, respectively. g :

[0129] Mapping pixel coordinates to a dimensionless standard space enables path planning results at different resolutions to be aligned and have a unified physical meaning, facilitating processing by all subsequent algorithms.

[0130] The lower limit protection constant is set to δ = 10. -6 As a lower limit protection, it prevents subsequent problems such as division by zero and numerical instability, and ensures that the solution remains numerically safe under extreme or degenerate conditions.

[0131] By acquiring the width and height of the field of view image and establishing a unified pixel coordinate system, the surgeon selects the pixel positions at both ends of a known physical scale in the image, calculates the conversion ratio between pixels and physical length, and uses this to determine the start and end positions of the endoscope tip and the glottis center. This solves the calibration error problem of traditional navigation systems under different endoscope resolutions, focal lengths, and installation angles. This method does not rely on a fixed calibration plate or pre-calibrated camera intrinsics; the surgeon can complete dynamic calibration online, shortening preoperative preparation time. Mapping pixel coordinates to a dimensionless standard space unifies the scale basis of subsequent algorithm processing, ensuring the consistency and comparability of path planning under different devices and resolutions. Simultaneously, by setting a lower limit protection constant, numerical instability problems under calibration anomalies or extreme values ​​are avoided. Compared to traditional methods relying on manual measurement or single automatic calibration techniques, this solution balances calibration accuracy and operational flexibility, and provides traceable physical mapping data, providing a solid physical foundation for subsequent navigation planning and visualization.

[0132] The process of normalizing the grayscale of each pixel in the image and calculating the gradient magnitude of each pixel based on the normalized grayscale values ​​specifically includes:

[0133] Obtain the original grayscale value at pixel coordinates (x, y), denoted as I(x, y);

[0134] Get the minimum grayscale value of all pixels I min =min 1≤x≤W,1≤y≤H I(x,y);

[0135] Get the maximum grayscale value of all pixels I max =max 1≤x≤W,1≤y≤H I(x,y);

[0136] Extract the upper and lower bounds of the current image grayscale to provide basic parameters for grayscale normalization, contrast enhancement and image segmentation;

[0137] Calculate the normalized gray value G(x,y) of pixel (x,y):

[0138] The original grayscale is standardized to the [0,1] range to eliminate the difference in grayscale scale under different acquisition conditions and improve the robustness of subsequent edge and structure recognition; in extreme cases, it is directly set to zero to ensure numerical safety.

[0139] Construct the gradient components in the x-direction respectively. gradient components in the y-direction

[0140]

[0141]

[0142] Accurately calculate the grayscale gradients in the x and y directions for each pixel, which are used for edge detection and obstacle structure determination. Differential protection is performed on boundary pixels to ensure that the gradient definition is complete.

[0143] Calculate the gradient magnitude M(x,y) of pixel (x,y):

[0144] By combining the gradients in the horizontal and vertical directions, the overall "intensity of change" of each pixel is obtained, providing a metric basis for obstacle segmentation and connected component analysis.

[0145] By performing interval mapping on the original grayscale values ​​of each pixel, the grayscale is standardized to a uniform range. Adaptive contrast adjustment is then performed using the minimum and maximum grayscale values ​​of the entire image, resolving grayscale drift and edge information loss issues caused by uneven illumination, reflections, or liquid occlusion in endoscopic images. Subsequently, the grayscale difference gradients in the horizontal and vertical directions are calculated separately and synthesized into the overall gradient magnitude of the pixel, providing a reliable feature measure for subsequent obstacle detection and structure localization. This grayscale normalization step automatically adapts to changes in the surgical environment without relying on manually set thresholds, ensuring consistency of image preprocessing results under different lighting conditions. The comprehensive gradient calculation method takes into account multi-directional information, improving the detectability of subtle anatomical boundaries and mucosal folds. Compared to methods relying solely on fixed thresholds or simple linear filtering, this approach maintains processing speed while improving robustness to key features, laying a high-quality foundation for accurate segmentation and obstacle avoidance.

[0146] The process involves setting a global threshold, classifying pixels above the threshold as obstacle regions, performing connected component analysis on the obstacle regions, and extracting the centroid coordinates of each connected component. Specifically, this includes:

[0147] Calculate the mean gradient magnitude of the entire image.

[0148] Calculate the standard deviation of gradient magnitude across the entire map.

[0149] The mean and standard deviation of the gradient of the entire image are obtained respectively, providing a reasonable quantitative basis for global adaptive threshold segmentation of obstacle pixels;

[0150] Set a global threshold T = μ for obstacle pixel detection. M +σ M The segmentation threshold is adaptively set using the "mean + standard deviation" method, which automatically adapts to different image complexities and achieves highly robust obstacle boundary detection.

[0151] Construct a label matrix Label(x,y)=0 and region number q=0; initialize the pixel label matrix and connected component numbers to prepare the data structure for subsequent connected component growth, region separation and obstacle analysis.

[0152] For each pixel (x,y), if M(x,y)≥T and Label(x,y)=0, execute steps S301 to S303:

[0153] S301, Update q←q+1;

[0154] S302. Initialize queue Q and enqueue (x,y), set Label(x,y) = q;

[0155] S302. When Q is not empty, dequeue (x′,y′), check the 4-neighborhood {(x′±1,y′),(x′,y′±1)}, for each neighboring point (u,v): if 1≤u≤W, 1≤v≤H, M(u,v)≥T and Label(u,v)=0, then set Label(u,v)=q and enqueue (u,v);

[0156] A breadth-first search is used to automatically group and classify pixels at all thresholds, identify and number all connected obstacle structures, and ensure that all obstacles can be tracked and processed individually.

[0157] Calculate the centroid of the j-th connected component in the pixel coordinate system.

[0158] Where j is the connected component number, j = {1,...,q}; Let x be the centroid coordinate of the j-th connected component; The y-coordinate of the centroid of the j-th connected component; |C j | represents the number of pixels in the j-th connected component; calculate the geometric center of each obstacle connected component, which serves as the core reference point for the obstacle repulsion field in the subsequent potential field algorithm, thereby improving obstacle avoidance accuracy;

[0159] Obtain the dimensionless coordinates of the j-th connected component. The centroid of the obstacle in the pixel coordinate system is standardized to ensure that the spatial position of the obstacle is strictly consistent with the coordinate system of the path planning.

[0160] By adaptively setting an obstacle segmentation threshold using the mean and standard deviation of the gradient magnitude across the entire image, pixels exceeding the threshold are automatically classified as candidate obstacle regions. A breadth-first connected component growth algorithm is then used to label and cluster these independent regions. Finally, the geometric center of each connected component is calculated and standardized, resolving the misclassification or omission issues caused by mucus accumulation or tissue folds in traditional methods within the endoscopic field of view. This adaptive threshold segmentation does not require a pre-set fixed threshold and can automatically adjust the segmentation sensitivity according to image complexity. Connected component analysis can segment obstacles of various shapes without shape priors, ensuring comprehensiveness and accuracy in obstacle detection. The extracted centroid coordinates provide precise spatial reference points for subsequent repulsive potential field construction. Compared to deep learning-based segmentation, this approach does not rely on a large number of training samples and offers higher real-time performance. Compared to simple edge or texture-based detection methods, it possesses stronger adaptability and stability, enabling it to adapt to the changing challenges of the clinical environment.

[0161] Based on the physical radius of the cannula tip, an attractive potential field function for the target point and a repulsive potential field function for each obstacle are established in the image space. The resulting composite potential field is obtained through function synthesis, specifically including:

[0162] Obtain the physical radius of the endotracheal intubation scope tip, denoted as R. tip Define the physical radius of the intubation endoscope to provide a physical basis for the barrier rejection area, ensuring the clinical safety and practical feasibility of the pathway planning;

[0163] Calculate the number of pixels R corresponding to the tip radius respectively. pix And the normalized value R of the tip radius:

[0164] The radius of the cannulation mirror tip is mapped to the pixel scale and normalized to establish an objective benchmark for the effective radius of the obstacle avoidance potential field.

[0165] Attractive potential field function U att (p) causes the trajectory to converge toward the target point: Where, ||·|| is the Euclidean norm; p = (p x ,p y ) represents the current dimensionless coordinates; p g Define the dimensionless coordinates of the target point; define the "gravity" that makes the path automatically tend toward the target point (glottis), encouraging the trajectory to approach the endpoint;

[0166] Calculate the distance d between the current point and the centroid of the obstacle. j =max(‖po j ‖,δ); where o j Let be the dimensionless coordinates of the j-th obstacle centroid; dynamically determine the safe distance between the current path point and each obstacle centroid and set a lower limit to prevent extreme proximity to the obstacle from causing the repulsive potential field to fail or the numerical value to explode;

[0167] Construct the repulsive potential function U of the j-th obstacle rep,j (p), to prevent the path from getting too close to obstacles:

[0168] Define a repulsive potential field with a limited range for each obstacle to ensure that the path automatically generates a strong repulsive force when it approaches the obstacle, and the repulsive force is zero outside the safe distance, thereby improving obstacle avoidance robustness and actual navigation safety.

[0169] Constructing the synthetic potential field By linearly superimposing the potential fields that attract the target and repel all obstacles, the overall path optimization target is obtained, ensuring that navigation can accurately reach the target while avoiding all obstacles.

[0170] By first obtaining the actual physical radius of the intubation endoscope tip and mapping it to image space, and then using this radius as the effective radius of the repulsive potential field, an attractive potential field for the target point and a repulsive potential field for each obstacle are constructed. These two fields are then linearly superimposed to form a composite potential field. This solves the safety hazard of previous algorithms ignoring the instrument width, which led to the planned path being too close to the airway wall or causing collisions. In this scheme, the attractive potential field ensures the path actively approaches the glottis target, while the repulsive potential field forms a protective zone around obstacles to maintain distance and achieve automatic avoidance. Introducing the physical radius ensures that the repulsive force gradually decays outside the safe distance range, avoiding excessive repulsion of the target point or getting stuck in a dead zone. Compared with single attractive or pure repulsive models, the potential field construction in this scheme is more in line with actual clinical needs, not only improving path smoothness but also providing more reliable navigation safety under complex airway geometry.

[0171] The calculation of the gradient of the synthetic potential field at any point, and the determination of the motion direction of the cannula tip based on this gradient, specifically includes:

[0172] Construct the gradient of the attractive potential at p Find the direct direction from the current position to the target point, which is used as the gradient component of the attractive potential field, indicating the direction of "fastest journey to the destination";

[0173] Construct the gradient of the j-th barrier repulsion potential field The direction and intensity of the obstacle repulsion potential field on the current position are given, reflecting the dynamic trend of path adjustment to avoid the obstacle;

[0174] Calculate the gradient of the composite potential field By integrating all factors of target attraction and obstacle repulsion, the system provides the optimal "descending potential field" direction for each path point, achieving automatic optimization.

[0175] Obtain the magnitude of the gradient of the synthetic potential field Quantifying the "downhill speed" of the potential field at the current position provides a precise basis for step size determination and termination conditions;

[0176] Construct the trajectory and plan the direction at point p. The gradient is standardized to a unit direction to ensure that each path update only reflects directional information and is not affected by the absolute value of the gradient. At the same time, the lower bound protects against unstable motion.

[0177] By calculating the gradient vector of the synthetic potential field at the current position, the attraction direction of the target and the repulsion direction of the obstacle are simultaneously quantified and fused, determining the optimal motion direction of the endoscope tip in real time. This solves the problems of slow response to intraoperative environmental changes and inability to dynamically avoid obstacles in traditional offline path planning. Specifically, the attraction gradient provides the most direct trend toward the target, while the repulsion gradient emits avoidance signals near obstacles. After local synthesis, the two form a single "downhill" direction, enabling the navigation path to continuously advance toward the target while flexibly avoiding newly appearing obstacles. This real-time gradient solution does not require a global search of the entire operating space; it only requires local calculation for a fast response, improving the path update frequency and system real-time performance. It is very suitable for applications with high timeliness requirements for surgical navigation.

[0178] The process of iteratively generating discrete trajectory points based on the direction of motion and a set step size, and determining path termination according to set conditions, specifically includes:

[0179] Set the maximum step size s for each iteration. max =R, maximum number of iterations The maximum distance to move forward in each step of the path is specified to prevent excessive jumps in the path, while the step limit is used to ensure that the algorithm terminates in extreme cases and that resources are safe;

[0180] Set the initial dimensionless coordinates as p0, the current iteration step k = 0, and the branch identifier string branch = NULL; set the starting point, step number, and branch identifier to ensure that the path growth algorithm is executed in an ordered and traceable state;

[0181] Repeat steps S601 to S606 until the exit condition is triggered:

[0182] S601. Calculate the gradient magnitude g at the k-th step. k =g(p k If g k If p ≤ δ, set branch = gradient_zero and exit; where p k The normalized coordinates of the k-th discrete trajectory point;

[0183] S602. Calculate the distance d between the k-th step and the target point. k =‖p k -p g ||; if d k ≤s max , let pk+1 =p g branch=reached_gold, exit;

[0184] S603, If k≥K max Set branch = max_iter, then exit;

[0185] S604. Otherwise, calculate p. k+1 =p k +s max d(p k );

[0186] S605, Boundary Constraints: p k+1,x p k+1,y Limited to [0,1];

[0187] S606, Let k ← k+1;

[0188] Each waypoint is automatically generated sequentially. This includes gradient detection, endpoint determination, step count limits, directional advancement, and boundary protection, ensuring the path always advances towards the endpoint and obstacle avoidance is effective.

[0189] S607, Branch Response:

[0190] If branch = gradient_zero, it indicates that the potential field gradient is approaching zero, which may be trapped in the extreme region, requiring manual intervention or reselection of the target.

[0191] If branch = max_iter, the message indicates that the number of iterations is too high and the path has not reached the destination. Please check for obstacles or adjust the parameters.

[0192] If branch = reached_gold, it indicates that the target has been reached and the path generation is complete;

[0193] For each stopping condition, detailed operation prompts are provided, such as being trapped in a local extremum, exceeding the step size and not reaching the endpoint, and successfully reaching the endpoint, to help the operator understand the current path planning status and intervene manually in a timely manner.

[0194] Output trajectory {p0,p1,...,p K}; where K is the step number of the last step; organize and output all discrete trajectory points to form a path data sequence for subsequent processing by the servo motor or visualization module.

[0195] By setting a maximum step size and a maximum number of iterations, and terminating path generation based on multiple criteria including the potential field gradient and distance to the target in each iteration, this mechanism provides explicit prompts for different exit scenarios. This addresses reliability issues such as navigation algorithms getting stuck in local maxima, excessive iterations, or infinite loops. Specifically, when the gradient strength falls below a safety threshold, it indicates a potential minima requiring manual intervention or target reselection; when the distance to the target reaches a preset threshold, it indicates successful arrival; and when the number of iterations exceeds the limit, it prompts for obstacle checks or parameter adjustments. Each scenario is accompanied by specific operational suggestions to help operators or upper-level systems quickly understand the planning status and respond accordingly. This multi-condition termination mechanism not only prevents uncontrolled path generation but also improves the system's stability and controllability in complex environments, providing higher navigation reliability and a better user experience for clinical use.

[0196] The process of calculating the displacement vector and steering angle for each segment of the generated discrete trajectory, and generating corresponding actuator commands based on the calculation results, specifically includes:

[0197] Construct the displacement vector Δp ​​of the k-th segment of the trajectory k =p k+1 -p k =(Δx) k ,Δy k ); where Δx k ,Δy k These are the x and y coordinate increments at the k-th step; calculate the motion vectors between continuous path points to provide basic data for steering angle calculation and servo command generation;

[0198] Calculate the turning angle θ of the k-th segment of the trajectory. k =atan2(Δy k ,Δx k ); where atan2(a,b) is the arctangent function, which returns the angle between the vector (b,a) and the x-axis; the displacement vector is converted into the angle command of the actual control servo motor to ensure that the physical control trajectory of the intubation mirror corresponds precisely to the planned path;

[0199] Each θ k The commands are packaged into servo steering protocol instructions and sent to the intubation endoscope execution system in real time. Each steering command is directly converted into a physical control signal to drive the intubation endoscope to achieve precise tube bending movements and complete the intubation guidance automatically or semi-automatically.

[0200] By vectorizing the spatial displacement between continuous discrete trajectory points and converting this vector into a corresponding steering angle, a control protocol command that can be directly invoked by actuators such as servos is generated. This solves the problems of mismatch between navigation algorithm output and mechanical execution interface, as well as the delays and errors caused by manual or secondary conversion. In this method, the incremental vector of trajectory points fully reflects the movement of the instrument, the steering angle accurately corresponds to the bending requirements of the endoscope, and the generated control command can be sent to the execution system in real time, ensuring that the algorithm-planned path and physical action are precisely matched. Compared with traditional solutions that only provide path coordinates or require external modules for translation, this solution achieves seamless integration from path planning to mechanical execution, improves the overall level of automation, shortens response time, and reduces the operator's workload and the risk of secondary errors.

[0201] The generated path and related guidance information are overlaid and displayed on the real-time acquired images to assist operators in performing intubation operations, specifically including:

[0202] Inverse normalization of coordinates: in, Here are the pixel coordinates corresponding to the k-th trajectory point; round(·) is the rounding function; p kx With p ky The x and y normalized coordinate components of the k-th trajectory point are used to restore the normalized trajectory point to image pixel coordinates, which facilitates accurate overlay and display on the real-time video stream.

[0203] In each frame of the image, connect all the lines sequentially. Path display is generated;

[0204] Highlight the next target point that is about to be reached using a dot;

[0205] The suggested intubation path and current position are displayed intuitively in the form of connecting lines, providing operators with clear and direct navigation prompts, improving the accuracy and safety of the operation;

[0206] Simultaneously, the branch prompt information obtained in step S607 is displayed on the visual interface; the path termination reason and branch information (such as getting stuck in a local extreme value, reaching the endpoint, or step overflow) are displayed in real time to help the operator quickly judge the current intubation navigation status and improve human-machine collaboration efficiency.

[0207] By denormalizing the normalized trajectory points, the path points are restored to pixel coordinates and visually displayed as lines on the real-time image stream. Simultaneously, the next target point is highlighted, and path termination or abnormality alerts are overlaid. This solves the cognitive burden and operational interruption problems caused by interface switching or information distraction during the operation. This visualization overlay seamlessly integrates navigation information with the original surgical field of view, allowing operators to simultaneously obtain the path, current location, and system prompts without taking their attention away, reducing operational risks. Compared to previous solutions that required viewing the path on an additional screen or separate interface, this solution provides integrated visual guidance, preserving detailed surgical images while integrating navigation information, improving user experience and surgical safety.

[0208] This embodiment also provides a system for a visually guided endotracheal intubation endoscope control method, including:

[0209] Image acquisition unit: used to acquire real-time field-of-view images;

[0210] Coordinate system calibration unit: used to establish the pixel coordinate system and calibrate the pixel with the physical scale;

[0211] Image processing unit: used to perform grayscale normalization, gradient calculation, obstacle segmentation, and centroid extraction;

[0212] Potential field construction and planning unit: used to construct the synthetic potential field and solve for the motion direction vector and trajectory generation;

[0213] Control command generation unit: used to calculate steering angle and generate servo control protocol commands;

[0214] Visualization display unit: Used to overlay path and prompt information onto the real-time field-of-view image.

[0215] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0216] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for controlling a visually guided endotracheal intubation endoscope, characterized in that, include: The field of view images acquired by the intubation endoscope are initialized, a two-dimensional pixel coordinate system with the upper left corner of the image as the origin is established, and the conversion relationship between pixels and actual physical length is determined through physical calibration. The image's pixels are normalized to grayscale, and the gradient magnitude of each pixel is calculated based on the normalized grayscale values. By setting a global threshold, pixels above the threshold are classified as obstacle regions. Connectivity analysis is then performed on the obstacle regions to extract the centroid coordinates of each connected region. Based on the physical radius of the tip of the cannula, the attractive potential field function of the target point and the repulsive potential field function of each obstacle are established in the image space, and the composite potential field is obtained by function synthesis. The gradient of the synthetic potential field at any point is calculated, and the direction of motion of the cannula tip is determined accordingly. Based on the direction of motion and the set step size, discrete trajectory points are generated iteratively, and path termination is determined according to the set conditions. The displacement vector and steering angle of each segment of the generated discrete trajectory are calculated, and the corresponding actuator commands are generated based on the calculation results. The generated path and related guidance information are overlaid on the real-time acquired images to assist operators in performing intubation procedures.

2. The method for controlling a visually guided endotracheal intubation endoscope according to claim 1, characterized in that, The initialization of the field-of-view image acquired by the intubation endoscope, establishing a two-dimensional pixel coordinate system with the upper left corner of the image as the origin, and determining the conversion relationship between pixels and actual physical length through physical calibration, specifically includes: Obtain the width and height of the current field of view image, denoted as W and H respectively; Taking the top left corner of the image as the origin O pix Establish a pixel coordinate system with the horizontal direction to the right as the positive x-axis and the vertical direction downward as the positive y-axis, with the unit being pixels and the pixel coordinates being (x, y). Calculate the length of the image diagonal The operator marks the pixel coordinates (x, y) at both ends of a known physical length ruler in the view. A ,y A ) and (x B ,y B ), and calculate the length N of the scale in the pixel coordinate system. ref : The actual physical length of the reference scale is denoted as L. ref ; Calculate the pixel-to-physical length conversion ratio. The surgeon selects the pixel coordinates of the endoscope tip and records them as follows: The surgeon selects the pixel coordinates of the center of the glottis, and records them as follows: Obtain the positions p0 and p1 of the intubation endoscope's starting and target points in the normalized coordinate system, respectively. g : The lower limit protection constant is set to δ = 10. -6 .

3. The method for controlling a visually guided endotracheal intubation endoscope according to claim 2, characterized in that, The process of normalizing the grayscale of each pixel in the image and calculating the gradient magnitude of each pixel based on the normalized grayscale values ​​specifically includes: Obtain the original grayscale value at pixel coordinates (x, y), denoted as I(x, y); Get the minimum grayscale value of all pixels I min =min 1≤x≤W,1≤y≤H I(x,y); Get the maximum grayscale value of all pixels I max =max 1≤x≤W,1≤y≤H I(x,y); Calculate the normalized gray value G(x,y) of pixel (x,y): Construct the gradient components in the x-direction respectively. gradient components in the y-direction Calculate the gradient magnitude M(x,y) of pixel (x,y):

4. The method for controlling a visually guided endotracheal intubation endoscope according to claim 3, characterized in that, The process involves setting a global threshold, classifying pixels above the threshold as obstacle regions, performing connected component analysis on the obstacle regions, and extracting the centroid coordinates of each connected component. Specifically, this includes: Calculate the mean gradient magnitude of the entire image. Calculate the standard deviation of gradient magnitude across the entire map. Set a global threshold T = μ for obstacle pixel detection. M +σ M ; Construct a label matrix Label(x,y) = 0 and a region number q = 0; For each pixel (x,y), if M(x,y)≥T and Label(x,y)=0, execute steps S301 to S303: S301, Update q←q+1; S302. Initialize queue Q and enqueue (x,y), set Label(x,y) = q; S302. When Q is not empty, dequeue (x′,y′), check the 4-neighborhood {(x′±1,y′),(x′,y′±1)}, for each neighboring point (u,v): if 1≤u≤W, 1≤v≤H, M(u,v)≥T and Label(u,v)=0, then set Label(u,v)=q and enqueue (u,v); Calculate the centroid of the j-th connected component in the pixel coordinate system. Where j is the connected component number, j = {1,...,q}; Let x be the centroid coordinate of the j-th connected component; The y-coordinate of the centroid of the j-th connected component; |C j | represents the number of pixels in the j-th connected component; Obtain the dimensionless coordinates of the j-th connected component.

5. The method for controlling a visually guided endotracheal intubation endoscope according to claim 4, characterized in that, Based on the physical radius of the cannula tip, an attractive potential field function for the target point and a repulsive potential field function for each obstacle are established in the image space. The resulting composite potential field is obtained through function synthesis, specifically including: Obtain the physical radius of the endotracheal intubation scope tip, denoted as R. tip ; Calculate the number of pixels R corresponding to the tip radius respectively. pix And the normalized value R of the tip radius: Attractive potential field function U att (p) causes the trajectory to converge toward the target point: Where, ||·|| is the Euclidean norm; p = (p x ,p y ) represents the current dimensionless coordinates; p g The coordinates of the target point are dimensionless. Calculate the distance d between the current point and the centroid of the obstacle. j =max(‖po j ‖,δ); where o j Let be the dimensionless coordinates of the centroid of the j-th obstacle; Construct the repulsive potential function U of the j-th obstacle rep,j (p), to prevent the path from getting too close to obstacles: Constructing the synthetic potential field 6. The method for controlling a visually guided endotracheal intubation endoscope according to claim 5, characterized in that, The calculation of the gradient of the synthetic potential field at any point, and the determination of the motion direction of the cannula tip based on this gradient, specifically includes: Construct the gradient of the attractive potential at p Construct the gradient of the j-th barrier repulsion potential field Calculate the gradient of the composite potential field Obtain the magnitude of the gradient of the synthetic potential field Construct the trajectory and plan the direction at point p.

7. The method for controlling a visually guided endotracheal intubation endoscope according to claim 6, characterized in that, The process of iteratively generating discrete trajectory points based on the direction of motion and a set step size, and determining path termination according to set conditions, specifically includes: Set the maximum step size s for each iteration. max =R, maximum number of iterations Set the initial dimensionless coordinates to p0, the current iteration step k = 0, and the branch identifier string branch = NULL; Repeat steps S601 to S606 until the exit condition is triggered: S601. Calculate the gradient magnitude g at the k-th step. k =g(p k If g k If p ≤ δ, set branch = gradient_zero and exit; where p k The normalized coordinates of the k-th discrete trajectory point; S602. Calculate the distance d between the k-th step and the target point. k =‖p k -p g ||; if d k ≤s max , let p k+1 =p g branch=reached_gold, exit; S603, If k≥K max Set branch = max_iter, then exit; S604. Otherwise, calculate p. k+1 =p k +s max d(p k ); S605, Boundary Constraints: p k+1,x p k+1,y Limited to [0,1]; S606, Let k ← k+1; S607, Branch Response: If branch = gradient_zero, it indicates that the potential field gradient is approaching zero, which may be trapped in the extreme region, requiring manual intervention or reselection of the target. If branch = max_iter, the message indicates that the number of iterations is too high and the path has not reached the destination. Please check for obstacles or adjust the parameters. If branch = reached_gold, it indicates that the target has been reached and the path generation is complete; Output trajectory {p0,p1,...,p K }; where K is the step number of the last step.

8. The method for controlling a visually guided endotracheal intubation endoscope according to claim 7, characterized in that, The process of calculating the displacement vector and steering angle for each segment of the generated discrete trajectory, and generating corresponding actuator commands based on the calculation results, specifically includes: Construct the displacement vector Δp ​​of the k-th segment of the trajectory k =p k+1 -p k =(Δx) k ,Δy k ); where Δx k ,Δy k These are the x and y coordinate increments at the k-th step, respectively; Calculate the turning angle θ of the k-th segment of the trajectory. k =atan2(Δy k ,Δx k ); where atan2(a,b) is the arctangent function, which returns the angle between the vector (b,a) and the x-axis; Each θ k The commands are packaged into servo steering protocol instructions and sent to the intubation endoscope execution system in real time.

9. The method for controlling a visually guided endotracheal intubation endoscope according to claim 8, characterized in that, The generated path and related guidance information are overlaid and displayed on the real-time acquired images to assist operators in performing intubation operations, specifically including: Inverse normalization of coordinates: in, Here are the pixel coordinates corresponding to the k-th trajectory point; round(·) is the rounding function; p kx With p ky For the k-th trajectory point, the x and y coordinate components are normalized. In each frame of the image, connect all the lines sequentially. Path display is generated; Highlight the next target point that is about to be reached using a dot; At the same time, the branch hint information obtained in step S607 is displayed on the visualization interface.

10. A system employing the visualization-guided endotracheal intubation endoscope control method of claim 9, characterized in that, include: Image acquisition unit: used to acquire real-time field-of-view images; Coordinate system calibration unit: used to establish the pixel coordinate system and calibrate the pixel with the physical scale; Image processing unit: used to perform grayscale normalization, gradient calculation, obstacle segmentation, and centroid extraction; Potential field construction and planning unit: used to construct the synthetic potential field and solve for the motion direction vector and trajectory generation; Control command generation unit: used to calculate steering angle and generate servo control protocol commands; Visualization display unit: Used to overlay path and prompt information onto the real-time field-of-view image.

Images