Visual trachea cannula and control method thereof

Abstract

The invention discloses a visual trachea cannula and a control method thereof.The visual trachea cannula comprises an insertion part, a camera arranged on the insertion part, a motor assembly used for pulling the camera to move in the circumferential direction and a controller connected with the camera and the motor assembly; the method comprises the following steps: acquiring an airway image after a trachea cannula is inserted into an airway; identifying a target foreign matter in the airway image, and calculating a foreign matter coordinate of the target foreign matter in an image coordinate system of the airway image; calculating a shielding degree and a shielding position of the target foreign matter in the airway image based on the foreign matter coordinates; determining the moving direction of the camera based on the shielding position and / or determining the moving step length of the camera based on the shielding degree; and based on the moving direction and / or the moving step length, controlling the motor assembly to pull the camera to avoid the obstacle. The camera can be quickly and accurately moved to a non-shielding area, a clear observation view angle is provided for surgical operators, and the intubation accuracy and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and in particular to a visual endotracheal cannula and a control method thereof. Background Art

[0002] In anesthesia surgery, endotracheal intubation through the oral or nasal cavity is a commonly used, rapid and effective tool for establishing an artificial airway. However, blind insertion during surgery can easily damage surrounding tissues, especially for novices. It is often difficult to position the intubation, difficult to operate, and cause serious damage from repeated intubation. It takes a long time, which artificially increases the risk of surgery and causes extremely serious complications and sequelae. Most ordinary endotracheal intubations are placed under the guidance of a laryngoscope, but in some patients, the glottis cannot be successfully placed if it is not fully exposed. Currently, there are some visual intubations on the market. However, when using a visual endotracheal intubation to intubate the airway, there may be sputum, blood clots, external objects and other foreign objects in the airway that block the camera, making it impossible to clearly observe the position in the body. At the same time, when the current visual intubation blocks the camera, it is difficult to get rid of the foreign object through manual operation, which makes it difficult to continue intubation.

[0003] Therefore, how to get rid of foreign body obstruction during visual endotracheal intubation and improve intubation efficiency and success rate has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the present invention provides a visual endotracheal intubation and a control method thereof to solve the technical problem of how to get rid of foreign body obstruction during the visual endotracheal intubation process and improve the intubation efficiency and success rate.

[0005] In a first aspect, the present invention provides a control method for a visual endotracheal cannula, the visual endotracheal cannula comprising an insertion portion, a camera disposed on the insertion portion, a motor assembly for pulling the camera to move in a circumferential direction, and a controller respectively connected to the camera and the motor assembly, the control method being applicable to the controller, the method comprising: Acquire an airway image after the endotracheal tube is inserted into the airway; Identifying a target foreign body in the airway image, and calculating the foreign body coordinates of the target foreign body in the image coordinate system of the airway image; Calculating the occlusion degree and occlusion position of the target foreign object in the airway image based on the foreign object coordinates; Determining a moving direction of the camera based on the occlusion position and / or determining a moving step length of the camera based on the occlusion degree, wherein the moving step length is positively correlated with the occlusion degree; Based on the moving direction and / or the moving step length, the motor assembly is controlled to pull the camera to avoid obstacles.

[0006] Optionally, the calculating the occlusion degree and occlusion position of the target foreign object in the airway image based on the foreign object coordinate value includes: Acquiring an image size of the airway image; Calculate the foreign object area of ​​the target foreign object based on the foreign object coordinates; Calculating the occlusion ratio of the target foreign object in the airway image as the occlusion degree based on the foreign object area and the image size; Determine the foreign object center coordinates of the target foreign object based on the foreign object coordinates; Calculating the positional relationship between the center coordinates of the foreign body and the image center coordinates of the airway image; Based on the positional relationship, position information of the target foreign object in the airway image is determined as the occlusion position.

[0007] Optionally, determining the moving direction of the camera based on the blocking position includes: Dividing the airway image into four quadrants based on the image center of the airway image; Determining the current quadrant in which the occluded position is located; The direction of a quadrant that is symmetrical to the current quadrant with respect to the image center is used as the moving direction.

[0008] Optionally, controlling the motor assembly to pull the camera to avoid obstacles based on the moving direction and / or the moving step length includes: Acquire a coordinate mapping relationship between a pre-established image coordinate system and the motor coordinate system; Based on the coordinate mapping relationship, the moving direction is mapped to the step direction of the motor, and the moving step length is mapped to the step value of the motor; The step direction and the step value are output to the motor to control the motor to pull the camera to move.

[0009] Optionally, controlling the motor assembly to pull the camera to avoid obstacles based on the moving direction and / or the moving step length includes: Determine the degree of dynamic occlusion in real time; Determine the dynamic movement step length using a step length adjustment function based on the dynamic occlusion degree; Based on the dynamic moving step length, the motor assembly is controlled in real time to pull the camera to avoid obstacles.

[0010] Optionally, the step size adjustment function is: step=base_step×(1+e (-k×(coverageθ)) ), Among them, base_step represents the basic moving step; coverageθ represents the degree of occlusion; K represents the adjustment coefficient, and step represents the dynamic moving step after adjustment.

[0011] Optionally, before determining the moving direction of the camera based on the occlusion position and / or determining the moving step length of the camera based on the occlusion degree, the method further includes: Determine whether the occlusion degree is greater than an occlusion threshold, and when the occlusion degree is greater than the occlusion threshold, enter the step of determining a moving direction of the camera based on the occlusion position and / or determining a moving step length of the camera based on the occlusion degree; It is determined whether the obstruction position is a preset central area of ​​the airway image. When the obstruction position is located in the preset central area, it is determined that it is difficult to effectively avoid obstacles, and self-cleaning is performed.

[0012] Optionally, during the obstacle avoidance process, each movement step is performed to obtain two airway images separated by N frames, where N≥1; Calculating the similarity between the two airway images; When the similarity is greater than the preset similarity, continue to perform obstacle avoidance until the similarity is continuously greater than the preset similarity after the preset number of movement steps, and output an alarm message; When the similarity is less than a preset value, it is determined that the obstacle avoidance is completed.

[0013] In a second aspect, the present invention provides a visual endotracheal intubation, comprising: an insertion portion, a camera arranged on the insertion portion, a motor assembly for pulling the camera to move in a circumferential direction, and a controller respectively connected to the camera and the motor assembly, wherein the controller executes the control method of the visual endotracheal intubation described in any one of the first aspects above.

[0014] Optionally, the motor assembly includes a motor and multiple steel wire ropes, one end of the multiple steel wire ropes are respectively connected to the motor, and the other ends pass through the tip of the insertion part and are circumferentially distributed and connected to the camera. The motor can pull each steel wire rope separately to achieve circumferential movement of the camera.

[0015] The present invention provides a visual endotracheal cannula and a control method thereof, wherein the visual endotracheal cannula comprises an insertion portion, a camera arranged on the insertion portion, a motor assembly for pulling the camera to move in a circumferential direction, and a controller respectively connected to the camera and the motor assembly, and the control method is applicable to the controller, and the method comprises: obtaining an airway image after the endotracheal cannula is inserted into the airway; identifying a target foreign body in the airway image, and calculating the foreign body coordinates of the target foreign body in the image coordinate system of the airway image; calculating the occlusion degree and occlusion position of the target foreign body in the airway image based on the foreign body coordinates; determining the moving direction of the camera based on the occlusion position and / or determining the moving step length of the camera based on the occlusion degree, wherein the moving step length is positively correlated with the occlusion degree; and controlling the motor assembly to pull the camera to avoid obstacles based on the moving direction and / or the moving step length. A camera is installed at the insertion part of the endotracheal tube, and the camera can be pulled by the motor assembly to move circumferentially. After the camera acquires the airway image, the target foreign body in the airway image is identified, and the occlusion position and occlusion degree of the target foreign body in the airway image are identified. The camera movement direction is determined based on the occlusion position, and the movement step length is determined based on the occlusion degree. Then, the motor assembly is controlled to pull the camera to move according to the determined movement direction and movement step length to avoid the target foreign body, so that the camera can be quickly and accurately moved to a non-obstructed area to obtain the best field of view, providing a clear observation angle for surgical operators to improve intubation accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a structural schematic diagram of a visual endotracheal intubation according to an embodiment of the present invention; Figure 2 is an enlarged schematic diagram of a local structure of a visual endotracheal tube according to an embodiment of the present invention; Figure 3 4 is a schematic diagram of a process of visual endotracheal intubation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0019] According to an embodiment of the present invention, a control method for visual endotracheal intubation is provided, such as Figure 1 and Figure 2 As shown, the visual endotracheal tube includes an insertion portion 10, a camera 20 arranged on the insertion portion 10, and a motor assembly 30 for pulling the camera 20 to move in a circumferential direction, wherein the motor assembly 30 includes a motor 31 and a plurality of steel wires 32, one end of each of the plurality of steel wires 32 is connected to the motor 31, and the other end passes through the tip of the insertion portion 10 and is circumferentially distributed and connected to the camera 20, and the motor 31 can pull each steel wire 32 respectively to achieve circumferential movement of the camera 20; in this embodiment, the steel wires 32 can be four, six, eight, etc., which can be evenly axially distributed on the camera 20, and the number of motors 31 can correspond to the steel wires 32 one by one, and the motors 31 are independent of each other, and then when the camera 20 moves circumferentially, each steel wire 32 is independent of each other and does not affect each other, and the moving direction of the camera 20 can be accurately controlled. Of course, in another embodiment, the number of motors 31 can also be less than the number of steel wires 32, and one motor 31 controls two or more steel wires 32. The controller is connected to the camera 20 and the motor 31 respectively. The camera 20 collects airway images and transmits them to the controller. The controller executes the following steps: Figure 3 The control method of the visual endotracheal intubation shown in the figure is to control the motor 31 to pull the camera 20 to move circumferentially to avoid obstacles of foreign objects in the airway. For details, see Figure 2 As shown, the control method of visual endotracheal intubation includes the following steps: S10. Acquire the airway image after the endotracheal tube is inserted into the airway. In this embodiment, after the visual endotracheal tube is inserted into the airway, the camera 20 acquires any frame of the video information of the insertion into the airway in real time as the airway image.

[0020] S20. Identify the target foreign body in the airway image, and calculate the foreign body coordinates of the target foreign body in the coordinate system of the airway image. In this embodiment, a target detection algorithm can be used to perform target detection on the airway image to identify the target foreign body in the airway image. Exemplarily, a YOLOv5 algorithm can be used as a target detection framework to train a target detection model for sputum, foreign matter, and blood clots. The collected airway image is input into the target detection model frame by frame or at intervals of preset frames to obtain the target foreign body detection result. In this embodiment, the target foreign body detection result can present the target foreign body boundary box in the airway image in the form of a boundary detection box. In other embodiments, for example, a convolutional neural network model can also be used for target foreign body detection.

[0021] In order to determine the foreign body coordinates of the target foreign body, in this embodiment, the image coordinate system of the airway image can be predefined. Specifically, with the upper left corner of the airway image as the origin, the airway image is normalized to a range of 0 to 1, so the leftmost position of the airway image corresponds to 0.0, the rightmost corresponds to 1.0, the top corresponds to 0.0, and the bottom corresponds to 1.0. For example, in an airway image, if a target foreign body is located on the left side of the airway image, then the horizontal coordinate img_x of the target foreign body in the image coordinate system of the airway image will be less than 0.5, while the horizontal coordinate img_x of the target foreign body located on the right side of the airway image is greater than 0.5. Therefore, after obtaining the target foreign body bounding box corresponding to the target foreign body, the foreign body coordinates are determined by calculating the coordinates of the target foreign body bounding box. In this embodiment, the foreign body coordinates may include the boundary coordinates of the target foreign body bounding box, and may also include the center coordinates of the target foreign body bounding box.

[0022] S30. Calculate the degree of occlusion and occlusion position of the target foreign object in the airway image based on the foreign object coordinates. In this embodiment, the foreign object center coordinates of the target foreign object can be determined based on the foreign object coordinates; calculate the positional relationship between the foreign object center coordinates and the image center coordinates of the airway image; and determine the position information of the target foreign object in the airway image as the occlusion position based on the positional relationship. In this embodiment, the occlusion position can be represented by the distance between the actual foreign object center coordinates and the image center, or by the orientation of the foreign object center relative to the image center.

[0023] Exemplarily, the airway image can be divided into four quadrants based on the image center of the airway image; the quadrant where the target foreign object is located is determined as the occlusion position of the target foreign object by calculating the positional relationship between the center coordinates of the foreign object and the center coordinates of the image. Specifically, firstly, the normalized coordinates of the center of the detected target foreign object bounding box are used to determine whether the target foreign object is located on the left or right side of the airway image based on whether the x coordinate in the center coordinates of the target foreign object bounding box is less than 0.5. Determine whether the target foreign object is located in the upper or lower part of the airway image based on whether the y coordinate in the center coordinates of the target foreign object bounding box is less than 0.5. Combining the left and right and top and bottom information, determine the quadrant where the target foreign object is located, and you can get the occlusion position of the target foreign object in the airway image.

[0024] The degree of occlusion can be characterized by the ratio of the foreign body area of ​​the target foreign body to the area of ​​the airway image. Specifically, the image size of the airway image is acquired; the foreign body area of ​​the target foreign body is calculated based on the foreign body coordinates; and the occlusion ratio of the target foreign body in the airway image is calculated based on the foreign body area and the image size as the degree of occlusion.

[0025] In one embodiment, for a smaller area of ​​occlusion, the impact on the field of view of the camera 20 is small, and there is no need to avoid the obstacle. For a larger area of ​​occlusion, the impact on the field of view of the camera 20 is large, so obstacle avoidance is required. In this embodiment, foreign body occlusion can be graded according to the degree of occlusion. For example, the occlusion threshold can be divided based on the occlusion ratio. When the occlusion ratio is greater than the occlusion threshold, it is considered that the current occlusion ratio has affected the field of view and the camera 20 needs to avoid the obstacle. When the occlusion ratio is less than the occlusion threshold, it is considered that the current occlusion ratio has little impact on the field of view, and there is no need to avoid the obstacle.

[0026] During the endotracheal intubation process, there may be some special situations. For example, when the target foreign object is in the center of the airway image and the occlusion ratio is large, it is difficult to achieve effective obstacle avoidance. For example, when the target foreign object is in the center of the airway image and the occlusion ratio is greater than 20%, (for example, the target foreign object is in the center of the airway image and the occlusion ratio of the target foreign object is greater than 20% of the area of ​​the airway image), and it lasts for multiple frames, it is considered difficult to achieve effective obstacle avoidance. Therefore, in this case, an alarm is triggered and the operation is stopped.

[0027] Specifically, for the target foreign object being in the preset center area, the method for judging whether it is difficult to achieve effective obstacle avoidance may include: obtaining the normalized coordinates (x, y) of the center of the detected target foreign object bounding box, and obtaining the size of the airway image. Based on the center coordinates of the target foreign object bounding box, determine whether the target foreign object bounding box is located in the preset center area of ​​the image (for example, the preset center area can be defined as a square or circular area that occupies 20% of the width of the airway image in the horizontal and vertical directions, and the x and y coordinates are both between 0.4 and 0.6). If so, it is considered that the target foreign object is located in the preset center area, and it is confirmed that it is difficult to effectively avoid obstacles.

[0028] Table 1 below provides an exemplary embodiment of the occlusion level classification:

[0029] S40. Determine the direction of movement of the camera head based on the occlusion position and / or determine the moving step of the camera 20 based on the occlusion degree, wherein the moving step is positively correlated with the occlusion degree. In one embodiment, since the motor 31 is required to pull the camera 20 to move circumferentially, it is necessary to establish a coordinate mapping relationship between the image coordinate system and the motor 31 coordinate system. Exemplarily, the coordinate values ​​in the image can be mapped to the step values ​​of the motor 31. Specifically, the image coordinate system can be aligned with the origin of the motor 31 through the center, the direction in the image is adapted to the movement direction of the motor 31, and the safety constraints on the movement range of the motor 31 are imposed, and the normalized coordinates are converted into step values ​​executable by the motor 31, so that the motor 31 moves accordingly according to the position of the object in the image, ensuring that the physical movement of the motor 31 corresponds precisely to the image position.

[0030] Exemplarily, the motion coordinate system of the motor 31 is mapped based on the normalized image coordinate system defined in the above embodiment to map the image coordinate value to the step value of the motor 31, and then the step value of the motor 31 is determined by the degree of occlusion of the target foreign object, and then the moving step length of each movement of the motor 31 to pull the camera 20 is determined. The moving direction of the motor 31 to pull the camera 20 is determined by the occlusion position of the target foreign object. For the motion coordinate system of the motor 31, the maximum stroke of the motor 31 is represented by motor_range. In this embodiment, the step value of the motor 31 is an integer, and the positive and negative values ​​determine the direction of movement. For example, in the horizontal direction, the step value of the motor 31 moving to the left is a negative number, and the step value to the right is a positive number; in the vertical direction, the step value of the motor 31 moving downward is a positive number, and the step value upward is a negative number.

[0031] The conversion between the image coordinate system and the motion coordinate system of the motor 31 is divided into horizontal mapping and vertical mapping. For the calculation of the step value in the horizontal direction, the center of the image is aligned with the origin of the motor 31: the center point of the image (0.5, 0.5) is mapped to the zero position of the motor 31 (0 step value). After that, the offset is calculated, that is, the image coordinate img_x of the airway image is subtracted by 0.5 (based on the image coordinate system defined in the above embodiment, the horizontal coordinate of the center position of the airway image is 0.5), so that the coordinate on the left becomes a negative value and the right becomes a positive value. After that, the range is expanded, and the coordinate after the image coordinate img_x is subtracted by 0.5 is multiplied by 2 to expand the range to the range of -1 to 1. Multiply it by the maximum stroke motor_range of the motor 31 and convert it into a step value with reference to the maximum stroke of the motor 31. Finally, the range of the step value is limited by the clamp function to ensure that it does not exceed the maximum stroke of the motor 31, that is, between -motor_range and motor_range. The processing in the vertical direction is similar to that in the horizontal direction, but the direction is different. Similarly, 0.5-img_y is calculated first to obtain the vertical coordinate value relative to the center. Because the top of the image is 0.5 and the bottom is 1.0, when img_y is less than 0.5, 0.5-img_y is a positive value, corresponding to the downward movement of the motor 31; when it is greater than 0.5, the result is a negative value, corresponding to the upward movement. The subsequent steps are the same as the horizontal direction. After range expansion and range limitation, the step value in the vertical direction is obtained. Among them, safety constraints must be set to ensure that the motor 31 does not move beyond its physical limits. Based on the clamp function, the step value is limited to between -motor_range and motor_range to avoid excessive movement of the camera 20, which may cause damage to the equipment or discomfort to the patient. The mapping of the image coordinate system and the motor motion coordinate system is completed in sequence, and a coordinate mapping relationship is established.

[0032] S50. Based on the moving direction and / or the moving step length, the motor assembly 30 is controlled to pull the camera 20 to avoid obstacles. In this embodiment, the coordinate mapping relationship between the pre-established image coordinate system and the motor coordinate system is obtained; based on the coordinate mapping relationship, the moving direction is mapped to the step direction of the motor, and the moving step length is mapped to the step value of the motor; the step direction and the step value are output to the motor to control the motor to pull the camera to move. Specifically: Regarding the moving direction, the moving direction of the camera 20 can be determined based on the quadrant where the target foreign body is located in the airway image. Specifically, the airway image can be divided into four quadrants based on the center of the airway image, namely, the upper left area, the upper right area, the lower left area, and the upper right area. The current quadrant where the target foreign body is located is determined based on the occlusion position of the target foreign body, and the direction of the quadrant that is symmetrical to the current quadrant relative to the center of the image is used as the moving direction. Specifically: According to whether the x-coordinate in the center coordinate of the target foreign body boundary box is less than the x-coordinate value of the center of the image, it is judged whether the target foreign body is located on the left or right side of the airway image. According to whether the y-coordinate in the center coordinate of the target foreign body boundary box is less than the y-coordinate value of the center of the image, it is judged whether the target foreign body is located in the upper part or the lower part of the airway image. The left and right and up and down judgment results are combined to determine the quadrant where the target foreign body is located. Using the direction mapping table, when the quadrant is the upper left area, it indicates that the camera 20 needs to move to the lower right to avoid the target foreign body. When the quadrant is the upper right area, it indicates that the camera 20 needs to move to the lower left to avoid the target foreign body. When the quadrant is the lower left area, it means that the camera 20 needs to move to the upper right to avoid the target foreign body. When the quadrant is the upper right area, it means that the camera 20 needs to move to the lower left to avoid the target foreign body. If the above quadrant is not matched, it means that the movable camera 20 maintains the current position.

[0033] With respect to the moving step length, the moving step length of the camera 20 is dynamically adjusted according to the degree of occlusion, and safety constraints are applied when necessary. Specifically, the dynamic occlusion degree is determined in real time; the dynamic moving step length is determined based on the dynamic occlusion degree using a step length adjustment function; and based on the dynamic moving step length, the motor assembly 30 is controlled in real time to pull the camera 20 to avoid obstacles.

[0034] Specifically, the following step length adjustment function may be used to determine the dynamic movement step length; step=base_step×(1+e (-k×(coverageθ)) ), Among them, base_step represents the basic moving step; coverageθ represents the degree of occlusion; K represents the adjustment coefficient, and step represents the dynamic moving step after adjustment.

[0035] In this embodiment, each time a moving step is completed, the airway image can be reacquired, the degree of occlusion and the occlusion position of the target foreign body can be identified again, and the moving step of the next movement is determined as the dynamic moving step based on the latest occlusion degree, and the moving direction of the next movement is determined as the dynamic moving direction based on the latest occlusion position. According to the dynamic moving step and the dynamic moving direction, the motor assembly 30 is controlled to pull the camera 20 to move to dynamically perform obstacle avoidance. In the process of obstacle avoidance, it can be achieved that when the degree of occlusion is less than the occlusion threshold, the increase of the moving step is relatively slow; when the degree of occlusion approaches or exceeds the occlusion threshold, the increase of the moving step will be significantly accelerated to quickly avoid the occlusion. Setting the maximum stroke of the motor 31 as the maximum allowable step, and using the clamp function to limit the moving step to the maximum allowable step, can prevent the movement of the camera 20 from exceeding the physical limit, causing equipment damage and patient discomfort.

[0036] In one embodiment, after the motor assembly pulls the camera 20 to move, a frame difference method is used to calculate the similarity between adjacent frames. When the similarity is less than a preset similarity, it indicates that the airway image has changed significantly, indicating that the obstacle avoidance is successful.

[0037] Exemplarily, during the obstacle avoidance process, each movement step is moved to obtain two airway images separated by N frames, where N≥1; the similarity between the two airway images is calculated; when the similarity is greater than a preset similarity, obstacle avoidance is continued until the similarity continues to be greater than a preset similarity after moving a preset number of movement steps, and an alarm message is output; when the similarity is less than a preset value, obstacle avoidance is determined to be completed.

[0038] After the movable camera 20 completes an obstacle avoidance movement, the system will immediately capture the current frame image and compare it with the previous frame image before obstacle avoidance. The structural similarity (SSIM) indicator is used to quantify the similarity between the two frames. The SSIM value ranges from 0 to 1, where 1 means that the two frames are exactly the same. The closer the value is to 1, the higher the structural similarity of the two frames. If the calculated SSIM value is greater than 0.95, the system believes that there is no significant change between the two frames. The movement of the camera 20 did not effectively avoid the target foreign object, and the obstacle avoidance operation may need to be reconsidered. If the obstacle avoidance is not successful, you can return to step S10 to perform obstacle avoidance again. When the target foreign object is still detected after multiple consecutive attempts at obstacle avoidance movement, an alarm is triggered. Stop the movement of the camera 20 to prevent further damage or risk. Upload the error log to the server or monitoring system for subsequent analysis and processing.

[0039] In one embodiment, in the process of controlling the camera 20 to avoid obstacles, when encountering abnormal situations, for example, the target foreign object is in the preset central area of ​​the airway image, the target foreign object is still detected after multiple obstacle avoidances, the motor 31 moves beyond the limit, and other abnormal situations, it is necessary to detect the abnormality in a targeted manner and execute corresponding measures to ensure that corresponding measures can be taken in time when encountering abnormalities to ensure safe and stable operation. Specifically, see the abnormality types shown in Table 2:

[0040] Among them, central region adhesion (E101): Trigger condition: When the system detects that the central area (defined as the 20% × 20% area in the center of the image) is continuously blocked for more than 3 seconds, the E101 alarm is triggered. Disposal measures: Sound and light alarm: Immediately activate the sound and light alarm to alert the operator. Start the self-cleaning program: The system will automatically start the self-cleaning program to try to clear the obstruction in the central area. The self-cleaning program may include air jets, vibrations, or other cleaning methods.

[0041] Multiple obstacle avoidance failures (E102): Trigger condition: When the system continues to detect an obstruction after multiple attempts to avoid obstacles, the E102 alarm is triggered. Disposal measures: Stop movement: The system will immediately stop the movement of the movable camera 20 to prevent further damage or risk. Upload error log: The system will upload the error log to the server or monitoring system for subsequent analysis and processing.

[0042] Mechanical overrun (E103): Trigger condition: When the system detects that the position of motor 31 exceeds the safe range, the E103 alarm is triggered. The safe range is usually determined by the maximum stroke and current position of motor 31. Disposal measures: Emergency stop: The system will immediately perform an emergency stop operation to stop all mechanical movements to prevent equipment damage or safety accidents.

[0043] The present application embodiment provides a visual endotracheal intubation, such as Figure 1 and Figure 2 As shown, it includes an insertion part 10, a camera 20 arranged on the insertion part 10, a motor assembly 30 for pulling the camera 20 to move in a circumferential direction, and a controller connected to the camera 20 and the motor assembly 30 respectively, wherein the controller includes a memory for a computer program; a processor for implementing the control method of the visual endotracheal intubation in the above-mentioned first aspect or any corresponding embodiment thereof when executing the computer program stored in the memory.

[0044] The motor assembly 30 includes a motor 31 and multiple steel wire ropes 32, one end of the multiple steel wire ropes 32 are respectively connected to the motors 31, and the other ends pass through the tip of the insertion part 10 and are circumferentially connected to the camera 20. The motor 31 can pull each steel wire rope 32 separately to achieve circumferential movement of the camera 20; in this embodiment, the steel wire ropes 32 can be four, six, eight, etc., which can be evenly distributed circumferentially on the camera 20. The number of motors 31 can correspond to the steel wire ropes 32 one by one, and the motors 31 are independent of each other. Therefore, when the camera 20 moves circumferentially, the steel wire ropes 32 are independent of each other and do not affect each other, so the movement direction of the camera 20 can be accurately controlled.

[0045] In one embodiment, the front end of the insertion portion 10 is separately provided with a tip portion and a rear end portion, the non-tip side of the tip portion and one side of the rear end portion are fixed at the same place of the endotracheal tube, the camera 20 is arranged on the unfixed side of the rear end portion, and the rear end portion is connected to a steel wire rope 32, which extends to the end of the endotracheal tube and is connected to a motor 31.

[0046] In another optional embodiment, the front end of the insertion part 10 is integrally formed, and a plurality of holes are provided on both sides of the tip of the insertion part 10 for passing a steel wire rope 32 therethrough. The steel wire rope 32 is connected to the camera 20, and the camera 20 is suspended on the inner side of the insertion part 10 and is only fixed by the steel wire rope 32. The steel wire rope 32 extends to the end of the endotracheal tube and is connected to the motor 31.

[0047] In another optional embodiment, four holes are provided at the top, bottom, left and right of the tip of the insertion portion 10, and each hole is penetrated by a steel wire 32, and one end of the four steel wires 32 is connected to the camera 20, so that the camera 20 can achieve 360° circumferential movement and achieve the best field of view adjustment. When the camera 20 recognizes that there is a foreign object or sputum blocking, it will be fed back to the controller, and a command will be given to the motor 31, and the motor 31 will control the steel wire 32 to move the position of the camera 20, so as to realize full automation of monitoring foreign objects and preventing screen blocking.

[0048] In another optional embodiment, the visual endotracheal tube further includes a cleaning connector 40 for injecting cleaning fluid. When performing self-cleaning, physiological saline can be injected through the cleaning connector 40 to clean foreign objects blocking the camera 20.

[0049] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0050] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers or network devices, etc.) to execute all or part of the steps of the method in the above embodiments.

[0051] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0052] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution provided in this embodiment.

[0053] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0054] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0055] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A visual endotracheal intubation control method, characterized in that: The visual endotracheal tube comprises an insertion portion, a camera disposed on the insertion portion, a motor assembly for pulling the camera to move in a circumferential direction, and a controller respectively connected to the camera and the motor assembly, wherein the control method is applicable to the controller, and the method comprises: Acquire an airway image after the endotracheal tube is inserted into the airway; Identifying a target foreign body in the airway image, and calculating the foreign body coordinates of the target foreign body in the image coordinate system of the airway image; Calculating the occlusion degree and occlusion position of the target foreign object in the airway image based on the foreign object coordinates; Determining a moving direction of the camera based on the occlusion position and / or determining a moving step length of the camera based on the occlusion degree, wherein the moving step length is positively correlated with the occlusion degree; Based on the moving direction and / or the moving step length, the motor assembly is controlled to pull the camera to avoid obstacles.

2. The control method of visual endotracheal intubation according to claim 1, characterized in that: The calculating the occlusion degree and / or occlusion position of the target foreign object in the airway image based on the foreign object coordinates comprises: Acquiring an image size of the airway image; Calculate the foreign object area of ​​the target foreign object based on the foreign object coordinates; Calculate the occlusion ratio of the target foreign object in the airway image as the occlusion degree based on the foreign object area and the image size; and / or Determine the foreign object center coordinates of the target foreign object based on the foreign object coordinates; Calculating the positional relationship between the center coordinates of the foreign body and the image center coordinates of the airway image; Based on the positional relationship, position information of the target foreign object in the airway image is determined as the occlusion position.

3. The control method of visual endotracheal intubation according to claim 2, characterized in that: The determining the moving direction of the camera based on the blocking position includes: dividing the airway image into four quadrants based on an image center of the airway image; Determining a current quadrant in which the occluded position is located; The direction of a quadrant that is symmetrical to the current quadrant with respect to the image center is used as the moving direction.

4. The control method of visual endotracheal intubation according to claim 1, characterized in that: The controlling the motor assembly to pull the camera to avoid obstacles based on the moving direction and / or the moving step length comprises: Acquire a coordinate mapping relationship between a pre-established image coordinate system and the motor coordinate system; Based on the coordinate mapping relationship, the moving direction is mapped to the step direction of the motor, and the moving step length is mapped to the step value of the motor; The step direction and the step value are output to the motor to control the motor to pull the camera to move.

5. The control method of visual endotracheal intubation according to claim 1 or 4, characterized in that: The controlling the motor assembly to pull the camera to avoid obstacles based on the moving direction and / or the moving step length comprises: Determine the degree of dynamic occlusion in real time; Determine the dynamic movement step length using a step length adjustment function based on the dynamic occlusion degree; Based on the dynamic moving step length, the motor assembly is controlled in real time to pull the camera to avoid obstacles.

6. The control method of visual endotracheal intubation according to claim 5, characterized in that: The step length adjustment function is: step=base_step×(1+e (-k×(coverageθ)) ), Among them, base_step represents the basic moving step; coverageθ represents the degree of occlusion; K represents the adjustment coefficient, and step represents the dynamic moving step after adjustment.

7. The control method of visual endotracheal intubation according to claim 2, characterized in that: Before determining the moving direction of the camera based on the occlusion position and / or determining the moving step length of the camera based on the occlusion degree, the method further includes: Determine whether the occlusion degree is greater than an occlusion threshold, and when the occlusion degree is greater than the occlusion threshold, enter the step of determining a moving direction of the camera based on the occlusion position and / or determining a moving step length of the camera based on the occlusion degree; It is determined whether the obstruction position is a preset central area of ​​the airway image. When the obstruction position is located in the preset central area, it is determined that it is difficult to effectively avoid obstacles, and self-cleaning is performed.

8. The control method of visual endotracheal intubation according to claim 1, characterized in that: During the obstacle avoidance process, each movement step is performed to obtain two airway images separated by N frames, where N ≥ 1; Calculating the similarity between the two airway images; When the similarity is greater than the preset similarity, the obstacle avoidance is continued until the similarity is continuously greater than the preset similarity after the preset number of movement steps, the obstacle avoidance is stopped, and an alarm message is output; When the similarity is less than a preset value, it is determined that the obstacle avoidance is completed.

9. A visual endotracheal intubation, characterized in that: include: An insertion portion, a camera arranged on the insertion portion, a motor assembly for pulling the camera to move in a circumferential direction, and a controller respectively connected to the camera and the motor assembly, wherein the controller executes the control method of the visual endotracheal intubation according to any one of claims 1 to 8.

10. The visual endotracheal tube according to claim 9, characterized in that: The motor assembly includes a motor and multiple steel wire ropes, one end of the multiple steel wire ropes are respectively connected to the motor, and the other ends pass through the tip of the insertion part and are circumferentially distributed and connected to the camera. The motor can pull each steel wire rope separately to achieve circumferential movement of the camera.

Images