A visual tracheal intubation and control method thereof

Abstract

The present invention discloses a visual endotracheal intubation and a control method thereof. The visual endotracheal intubation includes an insertion portion, a camera disposed on the insertion portion, a motor assembly for pulling the camera in a circumferential direction, and a controller connected to the camera and the motor assembly, respectively. The method comprises: obtaining an airway image after the endotracheal intubation is inserted into the airway; identifying a target foreign object in the airway image and calculating the foreign object coordinates of the target foreign object in an image coordinate system of the airway image; calculating the degree of occlusion and the occlusion position of the target foreign object in the airway image based on the foreign object coordinates; determining the camera's movement direction based on the occlusion position and / or the camera's movement step length based on the occlusion degree; and controlling the motor assembly to pull the camera for obstacle avoidance based on the movement direction and / or the movement step length. The method can quickly and accurately move the camera to an unobstructed area, providing a clear observation angle for the surgical operator and improving the accuracy and efficiency of intubation.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and in particular to a visual endotracheal intubation and a control method thereof. Background Art

[0002] During anesthesia surgery, endotracheal intubation through the mouth or nose is a commonly used, fast and effective tool for establishing an artificial airway, but blind insertion during surgery can easily damage surrounding tissues, especially for novices. It is often difficult to position the intubation, the operation is difficult, repeated intubation causes great damage, and it takes a long time, which artificially increases the risk of surgery and causes extremely serious complications and sequelae. Most ordinary endotracheal intubations are placed under the guidance of a laryngoscope, but in some patients, the glottis cannot be successfully placed if it is not fully exposed. There are some visual intubations on the market. However, when using a visual endotracheal intubation to intubate the airway, there may be sputum, blood clots, external objects and other foreign objects in the airway that block the camera, making it impossible to clearly observe the position in the body. At the same time, when the foreign object blocks the camera, it is difficult for the current visual intubation to get rid of the foreign object through manual operation, which makes it difficult to continue intubation.

[0003] Therefore, how to improve the visual endotracheal intubation process to get rid of foreign body obstruction and improve the intubation efficiency and success rate has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] In view of this, the present invention provides a visual endotracheal intubation and a control method thereof to solve the technical problem of how to improve the visual endotracheal intubation process by getting rid of foreign body obstruction and improving the intubation efficiency and success rate.

[0005] In a first aspect, the present invention provides a control method for a visual endotracheal tube, the visual endotracheal tube comprising an insertion portion, a camera disposed on the insertion portion, a motor assembly for pulling the camera to move in a circumferential direction, and a controller connected to the camera and the motor assembly, respectively. The control method is applicable to the controller and comprises:

[0006] Acquiring an airway image after the endotracheal tube is inserted into the airway;

[0007] Identifying a target foreign object in the airway image and calculating the foreign object coordinates of the target foreign object in the image coordinate system of the airway image;

[0008] Calculating the occlusion degree and occlusion position of the target foreign object in the airway image based on the foreign object coordinates;

[0009] Determining a movement direction of the camera based on the occlusion position and / or determining a movement step length of the camera based on the occlusion degree, wherein the movement step length is positively correlated with the occlusion degree;

[0010] The motor assembly is controlled to pull the camera to avoid obstacles based on the moving direction and / or the moving step size.

[0011] Optionally, the calculating the occlusion degree and occlusion position of the target foreign object in the airway image based on the foreign object coordinate value includes:

[0012] obtaining an image size of the airway image;

[0013] Calculating the foreign object area of ​​the target foreign object based on the foreign object coordinates;

[0014] Calculating an occlusion ratio of the target foreign object in the airway image as the occlusion degree based on the foreign object area and the image size;

[0015] Determine the foreign object center coordinates of the target foreign object based on the foreign object coordinates;

[0016] Calculating the positional relationship between the center coordinates of the foreign body and the image center coordinates of the airway image;

[0017] Based on the positional relationship, position information of the target foreign object in the airway image is determined as the blocking position.

[0018] Optionally, determining the movement direction of the camera based on the blocking position includes:

[0019] Dividing the airway image into four quadrants based on the image center of the airway image;

[0020] Determining the current quadrant in which the occlusion position is located;

[0021] The direction of a quadrant that is symmetrical to the current quadrant with respect to the image center is used as the moving direction.

[0022] Optionally, controlling the motor assembly to pull the camera to avoid obstacles based on the moving direction and / or the moving step size includes:

[0023] Acquiring a coordinate mapping relationship between a pre-established image coordinate system and the motor coordinate system;

[0024] Mapping the moving direction to the step direction of the motor based on the coordinate mapping relationship, and mapping the moving step length to the step value of the motor;

[0025] The step direction and the step value are output to the motor to control the motor to pull the camera to move.

[0026] Optionally, controlling the motor assembly to pull the camera to avoid obstacles based on the moving direction and / or the moving step size includes:

[0027] Determine the degree of dynamic occlusion in real time;

[0028] Determining a dynamic movement step length using a step length adjustment function based on the dynamic occlusion degree;

[0029] Based on the dynamic moving step length, the motor assembly is controlled in real time to pull the camera to avoid obstacles.

[0030] Optionally, the step size adjustment function is:

[0031] step=base_step×(1+e (-k×(coverageθ)) ),

[0032] Among them, base_step represents the basic moving step; coverageθ represents the degree of occlusion; K represents the adjustment coefficient, and step represents the dynamic moving step after adjustment.

[0033] Optionally, before determining the movement direction of the camera based on the occlusion position and / or determining the movement step length of the camera based on the occlusion degree, the method further includes:

[0034] Determining whether the occlusion degree is greater than an occlusion threshold, and when the occlusion degree is greater than the occlusion threshold, entering the step of determining a camera movement direction based on the occlusion position and / or determining a camera movement step size based on the occlusion degree;

[0035] It is determined whether the obstruction position is a preset central area of ​​the airway image. When the obstruction position is located in the preset central area, it is determined that it is difficult to effectively avoid the obstacle and self-cleaning is performed.

[0036] Optionally, during the obstacle avoidance process, each movement step is performed to obtain two airway images separated by N frames, where N ≥ 1;

[0037] Calculating the similarity between the two airway images;

[0038] When the similarity is greater than the preset similarity, the obstacle avoidance is continued until the similarity is continuously greater than the preset similarity after the preset number of movement steps, and an alarm message is output;

[0039] When the similarity is less than a preset value, it is determined that the obstacle avoidance is completed.

[0040] In a second aspect, the present invention provides a visual endotracheal intubation, comprising: an insertion portion, a camera arranged on the insertion portion, a motor assembly for pulling the camera to move in a circumferential direction, and a controller connected to the camera and the motor assembly respectively, wherein the controller executes the control method of the visual endotracheal intubation described in any one of the first aspects above.

[0041] Optionally, the motor assembly includes a motor and multiple steel ropes, one end of each of the steel ropes is connected to the motor, and the other end passes through the tip of the insertion part and is circumferentially distributed and connected to the camera. The motor can pull each steel rope separately to achieve circumferential movement of the camera.

[0042] The present invention provides a visual endotracheal cannula and a control method thereof, wherein the visual endotracheal cannula includes an insertion portion, a camera arranged on the insertion portion, a motor assembly for pulling the camera to move in a circumferential direction, and a controller respectively connected to the camera and the motor assembly, and the control method is applicable to the controller, and the method includes: obtaining an airway image after the endotracheal cannula is inserted into the airway; identifying a target foreign object in the airway image, and calculating the foreign object coordinates of the target foreign object in the image coordinate system of the airway image; calculating the occlusion degree and occlusion position of the target foreign object in the airway image based on the foreign object coordinates; determining the movement direction of the camera based on the occlusion position and / or determining the movement step length of the camera based on the occlusion degree, wherein the movement step length is positively correlated with the occlusion degree; and controlling the motor assembly to pull the camera to avoid obstacles based on the movement direction and / or the movement step length. A camera is installed at the insertion part of the endotracheal tube, and the camera can be pulled by the motor assembly to move circumferentially. After the camera obtains the airway image, it identifies the target foreign body in the airway image, and identifies the occlusion position and occlusion degree of the target foreign body in the airway image. The camera movement direction is determined based on the occlusion position, and the movement step length is determined based on the occlusion degree. Then, the motor assembly is controlled to pull the camera to move according to the determined movement direction and movement step length to avoid the target foreign body, so that the camera can be moved quickly and accurately to a non-obstructed area to obtain the best field of view, providing the surgical operator with a clear observation perspective, and improving the intubation accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 2. It is a schematic structural diagram of a visual endotracheal tube according to an embodiment of the present invention;

[0045] Figure 2 is an enlarged schematic diagram of a partial structure of a visual endotracheal tube according to an embodiment of the present invention;

[0046] Figure 34 is a flow chart of visual endotracheal intubation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0048] According to an embodiment of the present invention, a method for controlling a visual endotracheal intubation is provided. Figure 1 and Figure 2 As shown, the visual endotracheal tube includes an insertion portion 10, a camera 20 disposed on the insertion portion 10, and a motor assembly 30 for pulling the camera 20 in a circumferential direction. The motor assembly 30 includes a motor 31 and multiple steel cables 32. One end of each of the multiple steel cables 32 is connected to the motor 31, and the other end passes through the tip of the insertion portion 10 and is circumferentially connected to the camera 20. The motor 31 can pull each steel cable 32 to achieve circumferential movement of the camera 20. In this embodiment, the steel cables 32 can be four, six, eight, or other multiple steel cables 32 that can be evenly distributed axially on the camera 20. The number of motors 31 can correspond to the number of steel cables 32, and the motors 31 can be independent of each other. Therefore, when the camera 20 moves circumferentially, the steel cables 32 are independent of each other and do not affect each other, thereby accurately controlling the movement direction of the camera 20. Of course, in another embodiment, the number of motors 31 can be less than the number of steel cables 32, and one motor 31 can control two or more steel cables 32. The controller is connected to the camera 20 and the motor 31 respectively. The camera 20 collects airway images and transmits them to the controller. The controller executes the following steps: Figure 3 The control method of the visual endotracheal intubation is shown to control the motor 31 to pull the camera 20 to move circumferentially to avoid foreign objects in the airway. For details, see Figure 2 As shown, the control method of visual endotracheal intubation includes the following steps:

[0049] S10. Acquire an airway image after the endotracheal tube is inserted into the airway. In this embodiment, after the visual endotracheal tube is inserted into the airway, the camera 20 acquires any frame of the video information of the insertion into the airway in real time as the airway image.

[0050] S20. Identify the target foreign body in the airway image and calculate the foreign body coordinates of the target foreign body in the coordinate system of the airway image. In this embodiment, a target detection algorithm can be used to perform target detection on the airway image to identify the target foreign body in the airway image. Exemplarily, the YOLOv5 algorithm can be used as a target detection framework to train a target detection model for sputum, foreign bodies, and blood clots. The collected airway image is input into the target detection model frame by frame or at intervals of preset frames to obtain the target foreign body detection result. In this embodiment, the target foreign body detection result can be presented in the form of a boundary detection box in the airway image as a target foreign body boundary box. In other embodiments, for example, a convolutional neural network model can also be used to detect the target foreign body.

[0051] In order to determine the foreign body coordinates of the target foreign body, in this embodiment, the image coordinate system of the airway image can be predefined. Specifically, with the upper left corner of the airway image as the origin, the airway image is normalized to a range of 0 to 1, so the leftmost position of the airway image corresponds to 0.0, the rightmost corresponds to 1.0, the top corresponds to 0.0, and the bottom corresponds to 1.0. For example, in an airway image, if a target foreign body is located on the left side of the airway image, then the horizontal coordinate img_x of the target foreign body in the image coordinate system of the airway image will be less than 0.5, while the horizontal coordinate img_x of the target foreign body located on the right side of the airway image will be greater than 0.5. Therefore, after obtaining the target foreign body bounding box corresponding to the target foreign body, the foreign body coordinates are determined by calculating the coordinates of the target foreign body bounding box. In this embodiment, the foreign body coordinates may include the boundary coordinates of the target foreign body bounding box, or may include the center coordinates of the target foreign body bounding box.

[0052] S30. Calculate the degree of occlusion and occlusion position of the target foreign object within the airway image based on the foreign object coordinates. In this embodiment, the foreign object center coordinates of the target foreign object can be determined based on the foreign object coordinates; the positional relationship between the foreign object center coordinates and the image center coordinates of the airway image can be calculated; and based on the positional relationship, the position information of the target foreign object in the airway image can be determined as the occlusion position. In this embodiment, the occlusion position can be represented by the distance between the actual foreign object center coordinates and the image center, or by the orientation of the foreign object center relative to the image center.

[0053] Exemplarily, the airway image can be divided into four quadrants based on the image center of the airway image; the quadrant where the target foreign object is located is determined as the occlusion position of the target foreign object by calculating the positional relationship between the coordinates of the center of the foreign object and the coordinates of the image center. Specifically, first, the normalized coordinates of the center of the detected target foreign object bounding box are used to determine whether the target foreign object is located on the left or right side of the airway image based on whether the x coordinate in the center coordinate of the target foreign object bounding box is less than 0.5. Based on whether the y coordinate in the center coordinate of the target foreign object bounding box is less than 0.5, it is determined whether the target foreign object is located in the upper or lower part of the airway image. By combining the left and right and top and bottom information, the quadrant where the target foreign object is located is determined, and the occlusion position of the target foreign object in the airway image can be obtained.

[0054] The degree of occlusion can be represented by the ratio of the foreign body area of ​​the target foreign body to the area of ​​the airway image. Specifically, the image size of the airway image is obtained; the foreign body area of ​​the target foreign body is calculated based on the foreign body coordinates; and the occlusion ratio of the target foreign body in the airway image is calculated based on the foreign body area and the image size as the degree of occlusion.

[0055] In one embodiment, for smaller obstructions, the impact on the camera 20's field of view is minimal, and obstacle avoidance is not necessary. However, for larger obstructions, the impact on the camera 20's field of view is significant, and therefore obstacle avoidance is required. In this embodiment, foreign object obstructions can be graded based on the degree of obstruction. For example, an obstruction threshold can be set based on the obstruction ratio. When the obstruction ratio is greater than the obstruction threshold, it is considered that the current obstruction ratio has affected the field of view, and the camera 20 needs to avoid the obstacle. When the obstruction ratio is less than the obstruction threshold, it is considered that the current obstruction ratio has little impact on the field of view, and obstacle avoidance is not required.

[0056] During the tracheal intubation process, there may be some special situations. For example, when the target foreign object is in the center of the airway image and the occlusion ratio is large, it is difficult to achieve effective obstacle avoidance. For example, when the target foreign object is in the center of the airway image and the occlusion ratio is greater than 20% (for example, the target foreign object is in the center of the airway image and the occlusion ratio of the target foreign object is greater than 20% of the area of ​​the airway image), and it lasts for multiple frames, it is considered difficult to achieve effective obstacle avoidance. Therefore, in this case, an alarm is triggered and the operation is stopped.

[0057] Specifically, a method for determining whether effective obstacle avoidance is difficult due to the target foreign object being in a preset center area can include: determining the normalized coordinates (x, y) of the center of the detected target foreign object bounding box and the size of the airway image. Based on the center coordinates of the target foreign object bounding box, determining whether the target foreign object bounding box is located in the preset center area of ​​the image (for example, the preset center area can be defined as a square or circular area that occupies 20% of the width of the airway image in both the horizontal and vertical directions, with both x and y coordinates between 0.4 and 0.6). If so, the target foreign object is considered to be in the preset center area, and effective obstacle avoidance is confirmed to be difficult.

[0058] Table 1 below provides an exemplary embodiment of occlusion level classification:

[0059]

[0060] S40. Determine the direction of movement of the camera head based on the occlusion position and / or determine the moving step of the camera 20 based on the occlusion degree, wherein the moving step is positively correlated with the occlusion degree. In one embodiment, since the motor 31 is required to pull the camera 20 to move circumferentially, it is necessary to establish a coordinate mapping relationship between the image coordinate system and the motor 31 coordinate system. Exemplarily, the coordinate values ​​in the image can be mapped to the step values ​​of the motor 31. Specifically, the image coordinate system can be aligned with the origin of the motor 31 through the center, the direction in the image can be adapted to the movement direction of the motor 31, and the safety constraints on the movement range of the motor 31 can be imposed. The normalized coordinates can be converted into step values ​​executable by the motor 31, so that the motor 31 can move accordingly according to the position of the object in the image, ensuring that the physical movement of the motor 31 accurately corresponds to the image position.

[0061] Exemplarily, the motion coordinate system of motor 31 is mapped based on the normalized image coordinate system defined in the above embodiment to map the image coordinate values ​​to the step values ​​of motor 31. The step value of motor 31 is then determined by the degree of obstruction by the target foreign object, thereby determining the movement step length of each time motor 31 pulls camera 20. The direction of movement of motor 31 pulling camera 20 is determined by the location of the target foreign object obstruction. For the motion coordinate system of motor 31, the maximum range of motor 31 is represented by motor_range. In this embodiment, the step value of motor 31 is an integer, and the positive or negative value determines the direction of movement. For example, in the horizontal direction, the step value of motor 31 moving to the left is negative, while that of motor 31 moving to the right is positive. In the vertical direction, the step value of motor 31 moving downward is positive, while that of motor 31 moving upward is negative.

[0062] The conversion between the image coordinate system and the motor 31 motion coordinate system involves horizontal and vertical mapping. For horizontal step value calculation, the image center is aligned with the motor 31 origin: the image center point (0.5, 0.5) is mapped to the zero position of motor 31 (0 step value). Next, an offset is calculated by subtracting 0.5 from the image coordinate img_x of the airway image (based on the image coordinate system defined in the above embodiment, the horizontal coordinate of the center of the airway image is 0.5). This makes the left coordinate negative and the right coordinate positive. Next, the range is expanded by multiplying the image coordinate img_x by 2, extending the range to -1 to 1. This is then multiplied by the motor 31's maximum travel range, motor_range, to convert the step value to a step value referenced by the motor 31's maximum travel range. Finally, the clamp function is used to limit the step value range to ensure that it does not exceed the motor 31's maximum travel range, that is, between -motor_range and motor_range. The vertical processing is similar to the horizontal processing, but the direction is different. Similarly, 0.5-img_y is calculated first to obtain the vertical coordinate value relative to the center. Because the top of the image is 0.5 and the bottom is 1.0, when img_y is less than 0.5, 0.5-img_y is a positive value, corresponding to the downward movement of the motor 31; when it is greater than 0.5, the result is a negative value, corresponding to upward movement. The subsequent steps are the same as the horizontal direction. After range expansion and range limitation, the step value in the vertical direction is obtained. Safety constraints must be set to ensure that the motor 31 does not move beyond its physical limits. Based on the clamp function, the step value is limited to between -motor_range and motor_range to avoid excessive movement of the camera 20, which may cause damage to the equipment or discomfort to the patient. The mapping of the image coordinate system and the motor motion coordinate system is completed in sequence to establish a coordinate mapping relationship.

[0063] S50. Based on the movement direction and / or the movement step length, the motor assembly 30 is controlled to pull the camera 20 to avoid obstacles. In this embodiment, a coordinate mapping relationship between a pre-established image coordinate system and the motor coordinate system is obtained; based on the coordinate mapping relationship, the movement direction is mapped to the step direction of the motor, and the movement step length is mapped to the step value of the motor; the step direction and step value are output to the motor to control the motor to pull the camera to move. Specifically:

[0064] Regarding the direction of movement, the direction of movement of the camera 20 can be determined based on the quadrant in which the target foreign object is located in the airway image. Specifically, the airway image can be divided into four quadrants based on the center of the airway image, namely the upper left area, the upper right area, the lower left area, and the upper right area. The current quadrant in which the target foreign object is located is determined based on the occlusion position of the target foreign object, and the direction of the quadrant that is symmetrical with the current quadrant relative to the center of the image is used as the direction of movement. Specifically:

[0065] Based on whether the x-coordinate in the center coordinates of the target foreign body bounding box is less than the x-coordinate value of the image center, it is determined whether the target foreign body is located on the left or right side of the airway image. Based on whether the y-coordinate in the center coordinates of the target foreign body bounding box is less than the y-coordinate value of the image center, it is determined whether the target foreign body is located in the upper or lower part of the airway image. The left and right and upper and lower judgment results are combined to determine the quadrant where the target foreign body is located. Using the direction mapping table, when the quadrant is the upper left area, it indicates that the camera 20 needs to move to the lower right to avoid the target foreign body. When the quadrant is the upper right area, it indicates that the camera 20 needs to move to the lower left to avoid the target foreign body. When the quadrant is the lower left area, it indicates that the camera 20 needs to move to the upper right to avoid the target foreign body. When the quadrant is the upper right area, it indicates that the camera 20 needs to move to the lower left to avoid the target foreign body. If the above quadrants are not matched, it means that the movable camera 20 maintains its current position.

[0066] Regarding the moving step length, the moving step length of the camera 20 is dynamically adjusted according to the degree of occlusion, and safety constraints are applied when necessary. Specifically, the dynamic occlusion degree is determined in real time; the dynamic moving step length is determined using a step length adjustment function based on the dynamic occlusion degree; and based on the dynamic moving step length, the motor assembly 30 is controlled in real time to pull the camera 20 to avoid obstacles.

[0067] Specifically, the following step length adjustment function can be used to determine the dynamic movement step length:

[0068] step=base_step×(1+e (-k×(coverageθ)) ),

[0069] Among them, base_step represents the basic moving step; coverageθ represents the degree of occlusion; K represents the adjustment coefficient, and step represents the dynamic moving step after adjustment.

[0070] In this embodiment, each time a movement step is completed, the airway image can be reacquired, the degree of occlusion and the occlusion position of the target foreign object can be identified again, and the movement step of the next movement can be determined as the dynamic movement step based on the latest occlusion degree, and the movement direction of the next movement can be determined as the dynamic movement direction based on the latest occlusion position. According to the dynamic movement step and the dynamic movement direction, the motor assembly 30 is controlled to pull the camera 20 to move to dynamically perform obstacle avoidance. In the process of obstacle avoidance, it can be achieved that when the occlusion degree is less than the occlusion threshold, the increase in the movement step is relatively slow; when the occlusion degree approaches or exceeds the occlusion threshold, the increase in the movement step will be significantly accelerated to quickly avoid the occlusion. Setting the maximum stroke of the motor 31 as the maximum allowable step and using the clamp function to limit the movement step to within the maximum allowable step can prevent the movement of the camera 20 from exceeding the physical limit, causing damage to the equipment and discomfort to the patient.

[0071] In one embodiment, after the motor assembly pulls the camera 20 to move, a frame difference method is used to calculate the similarity between adjacent frames. When the similarity is less than a preset similarity, it indicates that the airway image has changed significantly, indicating that the obstacle avoidance is successful.

[0072] Exemplarily, during the obstacle avoidance process, two airway images N frames apart are obtained for each movement step, where N ≥ 1; the similarity between the two airway images is calculated; when the similarity is greater than a preset similarity, obstacle avoidance is continued until the similarity continues to be greater than the preset similarity after a preset number of movement steps, and an alarm message is output; when the similarity is less than a preset value, obstacle avoidance is determined to be completed.

[0073] After the movable camera 20 completes an obstacle avoidance movement, the system will immediately capture the current frame image and compare it with the previous frame image before obstacle avoidance. The structural similarity (SSIM) indicator is used to quantify the similarity between the two frames. The SSIM value ranges from 0 to 1, where 1 means that the two frames are exactly the same. The closer the value is to 1, the higher the structural similarity of the two frames. If the calculated SSIM value is greater than 0.95, the system believes that there is no significant change between the two frames. The movement of the camera 20 did not effectively avoid the target foreign object, and the obstacle avoidance operation may need to be reconsidered. If the obstacle avoidance is unsuccessful, return to step S10 and perform obstacle avoidance again. When the target foreign object is still detected after multiple consecutive attempts at obstacle avoidance movement, an alarm is triggered. Stop the movement of the camera 20 to prevent further damage or risk. Upload the error log to the server or monitoring system for subsequent analysis and processing.

[0074] In one embodiment, when controlling the camera 20 to avoid obstacles, if an abnormal situation is encountered, for example, a target foreign object is located in the preset center area of ​​the airway image, the target foreign object is still detected after multiple obstacle avoidance attempts, or the motor 31 moves beyond the limit, etc., it is necessary to specifically detect the abnormal situation and take corresponding measures to ensure that appropriate measures can be taken in a timely manner when an abnormal situation is encountered to ensure safe and stable operation. Specifically, see the abnormality types shown in Table 2:

[0075]

[0076] Among them, central region adhesion (E101):

[0077] Trigger Condition: When the system detects that the central area (defined as a 20% x 20% area in the center of the image) is continuously obscured for more than 3 seconds, the E101 alarm is triggered. Action: Audible and visual alarms immediately activate to alert the operator. Initiate Self-Cleaning: The system automatically initiates a self-cleaning process to attempt to clear obstructions in the central area. This self-cleaning process may include air jets, vibrations, or other cleaning methods.

[0078] Multiple Obstacle Avoidance Failures (E102): Trigger Condition: If the system detects an obstruction after multiple consecutive attempts at obstacle avoidance, the E102 alarm is triggered. Action: Stop Movement: The system immediately stops the movement of the movable camera 20 to prevent further damage or risk. Upload Error Log: The system uploads the error log to a server or monitoring system for subsequent analysis and processing.

[0079] Mechanical Overrun (E103): Triggering Condition: The E103 alarm is triggered when the system detects that the position of motor 31 is outside the safe range. The safe range is typically determined by the maximum travel and current position of motor 31. Action: Emergency Stop: The system immediately executes an emergency stop, halting all mechanical movement to prevent equipment damage or safety accidents.

[0080] The embodiment of the present application provides a visual endotracheal intubation, such as Figure 1 and Figure 2 As shown, it includes an insertion part 10, a camera 20 arranged on the insertion part 10, a motor assembly 30 for pulling the camera 20 to move in a circumferential direction, and a controller connected to the camera 20 and the motor assembly 30 respectively, wherein the controller includes a memory for a computer program; a processor for implementing the control method of visual endotracheal intubation in the above-mentioned first aspect or any corresponding embodiment thereof when executing the computer program stored in the memory.

[0081] The motor assembly 30 includes a motor 31 and multiple steel ropes 32, one end of each of the steel ropes 32 is connected to the motor 31, and the other end passes through the tip of the insertion part 10 and is circumferentially distributed and connected to the camera 20. The motor 31 can pull each steel rope 32 separately to realize the circumferential movement of the camera 20; in this embodiment, the steel ropes 32 can be four, six, eight, etc., which can be evenly distributed circumferentially on the camera 20. The number of motors 31 can correspond one-to-one to the steel ropes 32, and the motors 31 are independent of each other. When the camera 20 moves circumferentially, the steel ropes 32 are independent of each other and do not affect each other, and the movement direction of the camera 20 can be accurately controlled.

[0082] In one embodiment, the front end of the insertion portion 10 is separately provided with a tip portion and a rear end portion, the non-tip side of the tip portion and one side of the rear end portion are fixed at the same place of the endotracheal tube, the camera 20 is arranged on the unfixed side of the rear end portion, and the rear end portion is connected to a steel wire rope 32, which extends to the end of the endotracheal tube and is connected to the motor 31.

[0083] In another optional embodiment, the front end of the insertion part 10 is integrally formed, and multiple holes are provided on both sides of the tip of the insertion part 10 for passing a steel wire rope 32. The steel wire rope 32 is connected to the camera 20. The camera 20 is suspended inside the insertion part 10 and is fixed only with the steel wire rope 32. The steel wire rope 32 extends to the end of the tracheal tube and is connected to the motor 31.

[0084] In another optional embodiment, four holes are provided on the top, bottom, left and right sides of the tip of the insertion portion 10, each of which is threaded with a steel wire 32. One end of the four steel wires 32 is connected to the camera 20, allowing the camera 20 to achieve 360° circumferential movement, achieving optimal field of view adjustment. When the camera 20 recognizes that there is a foreign object or sputum blocking it, it will feedback to the controller, giving an instruction to the motor 31, which controls the steel wires 32 to move the position of the camera 20, achieving full automation of detecting foreign objects and preventing image obstruction.

[0085] In another optional embodiment, the visual endotracheal tube further includes a cleaning connector 40 for injecting cleaning fluid. When performing self-cleaning, physiological saline can be injected through the cleaning connector 40 to clean foreign matter blocking the camera 20.

[0086] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0087] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above-mentioned computer-readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes a number of instructions for causing one or more computer devices (such as personal computers, servers, or network devices) to execute all or part of the steps of the method in the above embodiments.

[0088] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is merely a logical functional division. In actual implementation, there may be other division methods, such as combining or integrating multiple units or components into another system, or ignoring or not implementing some features. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interface, indirect coupling or communication connection of units or modules, and may be electrical or other forms.

[0089] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected based on actual needs to achieve the purpose of the solution provided in this embodiment.

[0090] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0091] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0092] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A visual endotracheal intubation, characterized in that: include: An insertion portion, a camera disposed on the insertion portion, a motor assembly for pulling the camera to move in a circumferential direction, and a controller connected to the camera and the motor assembly respectively, wherein the controller performs the following control method including: Acquiring an airway image after the endotracheal tube is inserted into the airway; Identifying a target foreign object in the airway image and calculating the foreign object coordinates of the target foreign object in the image coordinate system of the airway image; Calculating the occlusion degree and occlusion position of the target foreign object in the airway image based on the foreign object coordinates; Determining a movement direction of the camera based on the occlusion position and / or determining a movement step length of the camera based on the occlusion degree, wherein the movement step length is positively correlated with the occlusion degree; The motor assembly is controlled to pull the camera to avoid obstacles based on the moving direction and / or the moving step size.

2. The visual endotracheal intubation according to claim 1, characterized in that: Calculating the occlusion degree and / or occlusion position of the target foreign object in the airway image based on the foreign object coordinates includes: obtaining an image size of the airway image; Calculating the foreign object area of ​​the target foreign object based on the foreign object coordinates; Calculating the occlusion ratio of the target foreign object in the airway image as the occlusion degree based on the foreign object area and the image size; and / or Determine the foreign object center coordinates of the target foreign object based on the foreign object coordinates; Calculating the positional relationship between the center coordinates of the foreign body and the image center coordinates of the airway image; Based on the positional relationship, position information of the target foreign object in the airway image is determined as the blocking position.

3. The visual endotracheal intubation according to claim 2, characterized in that: The determining the moving direction of the camera based on the blocking position includes: dividing the airway image into four quadrants based on an image center of the airway image; Determining the current quadrant in which the occlusion position is located; The direction of a quadrant that is symmetrical to the current quadrant with respect to the image center is used as the moving direction.

4. The visual endotracheal intubation according to claim 1, characterized in that: The controlling the motor assembly to pull the camera to avoid obstacles based on the moving direction and / or the moving step size includes: Obtain the coordinate mapping relationship between the pre-established image coordinate system and the motor coordinate system; Mapping the moving direction to the step direction of the motor based on the coordinate mapping relationship, and mapping the moving step length to the step value of the motor; The step direction and the step value are output to the motor to control the motor to pull the camera to move.

5. The visual endotracheal intubation according to claim 1 or 4, characterized in that: The controlling the motor assembly to pull the camera to avoid obstacles based on the moving direction and / or the moving step size includes: Determine the degree of dynamic occlusion in real time; Determining a dynamic movement step length using a step length adjustment function based on the dynamic occlusion degree; Based on the dynamic moving step length, the motor assembly is controlled in real time to pull the camera to avoid obstacles.

6. The visual endotracheal intubation according to claim 5, characterized in that: The step size adjustment function is: step=base_step×(1+e(-k×(coverageθ))), Among them, base_step represents the basic moving step; coverageθ represents the degree of occlusion; K represents the adjustment coefficient, and step represents the dynamic moving step after adjustment.

7. The visual endotracheal intubation according to claim 2, characterized in that: Before determining the moving direction of the camera based on the occlusion position and / or determining the moving step length of the camera based on the occlusion degree, the method further includes: Determining whether the occlusion degree is greater than an occlusion threshold, and when the occlusion degree is greater than the occlusion threshold, entering the step of determining a camera movement direction based on the occlusion position and / or determining a camera movement step size based on the occlusion degree; It is determined whether the obstruction position is a preset central area of ​​the airway image. When the obstruction position is located in the preset central area, it is determined that it is difficult to effectively avoid the obstacle and self-cleaning is performed.

8. The visual endotracheal intubation according to claim 1, characterized in that: During the obstacle avoidance process, each movement step is performed to obtain two airway images separated by N frames, where N ≥ 1; Calculating the similarity between the two airway images; When the similarity is greater than the preset similarity, the obstacle avoidance is continued until the similarity is continuously greater than the preset similarity after the preset number of movement steps, the obstacle avoidance is stopped, and an alarm message is output; When the similarity is less than a preset value, it is determined that the obstacle avoidance is completed.

9. The visual endotracheal tube according to claim 1, characterized in that: The motor assembly includes a motor and multiple steel ropes, one end of each of the steel ropes is connected to the motor, and the other end passes through the tip of the insertion part and is circumferentially distributed and connected to the camera. The motor can pull each steel rope separately to achieve circumferential movement of the camera.

Images