Intelligent catheterization robot automatic registration and self-adaptive path planning method and device

Abstract

The application provides an intelligent urinary catheterization robot automatic registration and self-adaptive path planning method and device, proposes a GLM-YOLO model, adopts a global local module to replace a C2F module in a traditional YOLO model, accelerates inference speed, focuses on local key information, realizes accurate identification and segmentation of a urethra, and simultaneously, the system performs position information registration according to an identification result, so that a target visual field is always in a target center position, so as to accurately control a soft lens to enter the urethra, in the process, according to a signal output by a ring force feedback sensor, the force applied by the front end of the urinary catheter is adjusted, so as to ensure that the urethra wall is not excessively pressed when the urinary catheter enters the urethra. Finally, the system is provided with a database and a path memory (especially important in the case of limited single-hole operation visual field) function, different urinary catheterization paths and coordinates can be adopted according to different users, automatic urinary catheterization is realized, and the repetitive work of patients is reduced.

Description

Technical Field

[0001] This invention relates to the field of flexible catheter technology, and particularly to a method and apparatus for automatic registration and adaptive path planning of an intelligent catheterization robot. Background Technology

[0002] In modern life, various factors such as trauma, congenital diseases, and tumors can cause spinal cord nerve damage, leading to lower limb movement disorders, paraplegia, and bowel and bladder dysfunction, especially urinary dysfunction. This type of urinary dysfunction caused by nerve damage is commonly referred to as neurogenic bladder.

[0003] Neurogenic bladder, which has a huge impact on the quality of life of people with disabilities, requires patients to carry a variety of tools such as urinary devices, diapers, and catheters, which seriously affects their lives and social interactions. Moreover, improper personal management can lead to malignant changes such as hydronephrosis and renal failure. These inconveniences and non-professional factors have become important reasons affecting their lives.

[0004] A significant number of people with disabilities require urinary catheters or intermittent self-catheterization. However, due to urination difficulties and a sense of privacy, these issues often go unnoticed by society. Patients often find catheterization unsightly and the repeated self-catheterization troublesome and inconvenient, allowing their condition to worsen. This lack of social care and support, coupled with the patients' own sense of shame, prevents their condition from improving and often leads to further deterioration.

[0005] To address this issue, the First Affiliated Hospital of Xi'an Jiaotong University School of Medicine proposed a magnetically coupled male intelligent catheterization robot system (Publication No.: CN111467117A). This system, through a specific structural design, avoids direct contact between nurses and male patients' genitals throughout the entire process and achieves sterile and stable catheterization, effectively alleviating the psychological distress of female nurses. However, this device requires professional operation, lacks image-assisted judgment, and is relatively complex to operate. Furthermore, its fixed components strictly fix the catheterization site, preventing movement and potentially causing psychological stress and reduced comfort for the patient. Summary of the Invention

[0006] To achieve the above-mentioned objectives and other advantages of the present invention, a first objective of the present invention is to provide an automatic registration and adaptive path planning method for an intelligent catheterization robot, comprising the following steps:

[0007] Acquire image data of the tip of the patient's urinary catheter tissue;

[0008] The anterior region of the catheter tissue to be identified in the image data is labeled;

[0009] The GLM-YOLO target segmentation network is used to detect and identify the front region of the catheter tissue in real time, and the identified region information is fed back.

[0010] Based on the urethral orifice segmentation information fed back in real time by the GLM-YOLO target segmentation network, the centroid of the urethral orifice segmentation region is found. The centroid of the urethral orifice segmentation region is compared with the center point of the endoscope image to determine the position of the centroid. The flexible catheter endoscope is then registered with the position of the centroid through the three-axis control system of the flexible catheter endoscope.

[0011] The pressure of the urethral wall on the flexible catheter is detected in real time by a force feedback sensor. The detected pressure is used to determine whether the force applied to the urethral wall by the flexible catheter during its movement is safe.

[0012] The optimal catheterization route is planned based on the assessment results;

[0013] It stores data corresponding to different users and records the path information of each catheterization. By comparing multiple data, it generates automatic catheterization configuration parameters.

[0014] Furthermore, after the step of annotating the anterior contour of the urinary catheter tissue and the urinary catheter orifice in the image data, the following steps are also included:

[0015] The labeled user data is enhanced by Gaussian blur, rotation, brightness, contrast, saturation, and hue processing.

[0016] Furthermore, the step of real-time detection and identification of the anterior region of the catheter tissue using the GLM-YOLO target segmentation network includes the following steps:

[0017] Feature extraction is performed on the output feature maps from the three high-resolution spatial branches;

[0018] Feature maps from three different resolutions are merged along the channel dimension and then used through an axial attention mechanism to obtain more comprehensive information.

[0019] The output feature maps from the context branch are processed by the GLM-YOLO object segmentation network and then upsampled to a high resolution level by bilinear interpolation to generate semantic weights.

[0020] The GLM-YOLO target segmentation network multiplies the semantic weights from the context branch with the high-resolution features from the spatial branch element-wise, enabling low-level spatial features to acquire richer high-level semantic information.

[0021] Furthermore, the axial attention mechanism is expressed as:

[0022]

[0023] in, , Represents tensor Arranged according to a given dimension, Represents a vector in which all elements are equal to 1, positional embedding. , for linear projection, , , , Given the input feature map, and , For learning weights, These represent the height, width, and number of channels of the feature map, respectively.

[0024] Furthermore, the three-axis control system for the flexible catheter endoscope includes an X-axis motor, a Y-axis motor, a Z-axis motor, and a control unit; wherein,

[0025] The X-axis motor is used to control the horizontal movement of the urinary catheter;

[0026] The Y-axis motor is used to control the vertical movement of the urinary catheter;

[0027] The Z-axis motor is used to control the movement of the catheter in the forward and backward directions;

[0028] The control unit is used to receive motion commands transmitted by the system and coordinate and control the drives of the X-axis motor, Y-axis motor, and Z-axis motor.

[0029] Furthermore, the flexible catheter endoscope is equipped with a high-resolution camera and an illumination system at its tip, and the flexible catheter endoscope is used to provide clear imaging of the patient's urethra and bladder.

[0030] The process of aligning the flexible catheter endoscope with the center of mass using the three-axis control system of the flexible catheter endoscope includes the following steps:

[0031] The urethral orifice segmentation result is obtained by detecting the image acquired by the flexible catheter endoscope using the GLM-YOLO segmentation algorithm.

[0032] If a urethral opening segmentation result exists in the current frame, calculate the contour of the urethral opening segmentation result, calculate the centroid of the urethral opening segmentation result, and determine the position of the centroid.

[0033] Furthermore, the registration of the flexible catheter endoscope with the center of mass via the three-axis control system includes the following steps:

[0034] Using the coordinates of the image center point Establish a rectangular coordinate system with the origin. This system divides the image into four intervals, starting from the top left corner and proceeding clockwise: interval one, interval two, interval three, and interval four. , , Image height, Image width;

[0035] Using the coordinates of the center point of the endoscopic image Establish a rectangular coordinate system with the origin and calculate the centroid. Distances to the horizontal axis, vertical axis, and origin: , , ;

[0036] Determine the center of mass The specific rules for determining the location are as follows: The specification also stipulates that the three-axis motors of the flexible catheter endoscope drive the endoscope in three directions: lateral movement (X), longitudinal movement (Y), and depth movement (Z). The lateral movement (X) includes two actions: X to the left and X to the right; the longitudinal movement (Y) includes two actions: Y up and Y down; and the depth movement (Z) includes two actions: Z forward and Z backward. This is based on the center of mass. The specific drive method of the three-axis motor at the location is as follows:

[0037]

[0038] The magnitudes of the lateral motion X and the longitudinal motion Y are: and , It is a manually set threshold, when the centroid Coordinates of the image center point The distance between them is less than the threshold At this point, it is considered that the automatic registration of the target position has been completed, and the flexible catheter is then driven forward to enter the urethra.

[0039] Furthermore, the force feedback sensor is a ring force feedback sensor, which is located at the front end of the catheter. The ring force feedback sensor is used to monitor the force on the front end of the catheter in real time, including its magnitude and direction.

[0040] The step of determining whether the force applied to the urethral wall by the flexible catheter endoscope during its movement is safe by detecting pressure includes the following steps:

[0041] When the signal output by the ring force feedback sensor reaches or exceeds the safety threshold, adjust the position of the flexible catheter endoscope to reduce the pressure exerted by the catheter tip on the pressure point.

[0042] Furthermore, the step of planning the optimal catheter route based on the judgment result includes the following steps:

[0043] After automatic registration with the target position, when the flexible catheter endoscope advances to the urethral opening, the current position is marked as the starting position;

[0044] By adjusting the position of the tip of the flexible catheter endoscope, the forward movement of the flexible catheter endoscope is controlled while keeping the force on the tip of the flexible catheter endoscope within a safe threshold.

[0045] The adaptive pathfinding algorithm calculates and compares all pressure values ​​on the ring force feedback sensor, and adjusts the navigation trajectory in conjunction with the force feedback control system to achieve force balance. At the same time, the adaptive pathfinding algorithm optimizes the trajectory to ensure that the catheter insertion process is smooth and continuous.

[0046] The catheter insertion is terminated when the urethral endoscope is successfully positioned in the bladder or when the adaptive pathfinding algorithm detects an abnormality.

[0047] Furthermore, the process of storing data corresponding to different users and recording the path information of each catheterization, and generating automatic catheterization configuration parameters through multiple data comparisons, includes the following steps:

[0048] Use a database to determine the user;

[0049] If the user's historical information exists, the detailed technical indicators of each catheterization are processed and calculated, and automatic catheterization is initiated.

[0050] If no historical information exists, the user's name, gender, and age will be saved. At the same time, detailed technical indicators of each catheterization will be recorded, including the size of the urethral opening, the action commands of the three-axis control system of the flexible catheter after it enters the urethra, the real-time data of the force feedback sensor, and the total time taken for the catheter to reach the target position.

[0051] Furthermore, the step of initiating automatic catheterization includes the following steps:

[0052] Based on the established database, find all the information of the current user, integrate and compare the comprehensive data of each catheterization process, and filter out the normal catheterization process;

[0053] By comparing and analyzing the catheterization path and real-time data from the force feedback sensor during each catheterization, the optimal catheterization path is calculated.

[0054] Automatic catheterization is performed based on the optimal catheterization path.

[0055] The second objective of this invention is to provide an automatic registration and adaptive path planning device for an intelligent catheterization robot, which implements the aforementioned method, including an image acquisition module, an image processing module, a GLM-YOLO target segmentation network module, a three-axis control system for a flexible catheter endoscope, an automatic target position registration module, a force feedback control system, an adaptive pathfinding algorithm module, and a database management and automatic catheterization module; wherein,

[0056] The image acquisition module is used to acquire image data of the front end of the patient's urinary catheter tissue;

[0057] The image processing module is used to annotate the anterior region of the catheter tissue to be identified in the image data;

[0058] The GLM-YOLO target segmentation network module is used to detect and identify the front end region of the catheter tissue in real time and provide feedback on the identified region information.

[0059] The three-axis control system of the flexible catheter endoscope has three control axes, including the horizontal X-axis, the vertical Y-axis and the depth Z-axis. The control axes are used to control the movement direction and advance depth of the flexible catheter endoscope.

[0060] The automatic registration module for the target position is used to find the centroid of the urethral orifice segmentation region based on the real-time feedback of the urethral orifice segmentation information from the GLM-YOLO target segmentation network. The centroid of the urethral orifice segmentation region is compared with the center point of the endoscope image to determine the position of the centroid. The flexible catheter endoscope is then registered with the position of the centroid through the three-axis control system of the flexible catheter endoscope.

[0061] The force feedback control system is used to detect the pressure of the urethral wall on the flexible catheter in real time through a force feedback sensor, and to determine whether the force applied by the flexible catheter to the urethral wall during the movement of the catheter is safe by using the detected pressure.

[0062] The adaptive pathfinding algorithm module is used to plan the optimal catheter path based on the judgment result;

[0063] The database management and automatic catheterization module is used to store data corresponding to different users, and record the path information of each catheterization. By comparing multiple data, automatic catheterization configuration parameters are generated.

[0064] Compared with the prior art, the beneficial effects of the present invention are:

[0065] This invention provides an automatic registration and adaptive path planning method and device for an intelligent catheterization robot. It proposes a GLM-YOLO model, replacing the C2F module in the traditional YOLO model with a global-local module to accelerate inference and focus on key local information, achieving accurate urethral identification and segmentation. Simultaneously, the system performs positional registration based on the identification results to ensure the target field of view remains at the bullseye, facilitating precise insertion of the flexible endoscope into the urethra. During this process, the force applied to the catheter tip is adjusted based on signals from a ring force feedback sensor to prevent excessive compression of the urethral wall upon catheter entry. Finally, the system incorporates a database and path memory function (particularly important in situations with limited field of view during single-port operation), allowing for automated catheterization using different paths and coordinates for different users, reducing repetitive work for patients.

[0066] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description

[0067] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0068] Figure 1 This is a flowchart of the automatic registration and adaptive path planning method for the intelligent urinary catheterization robot in Example 1;

[0069] Figure 2 Flowchart of the intelligent urinary catheterization robot operation;

[0070] Figure 3 This is a screenshot of the software interface for an intelligent urinary catheterization robot.

[0071] Figure 4 Here is a diagram of the GLM-YOLO network structure;

[0072] Figure 5 A schematic diagram of the axial attention mechanism;

[0073] Figure 6 This is a schematic diagram of the structure of an intelligent urinary catheterization robot;

[0074] Figure 7 This is a diagram showing the AI ​​detection results and a magnified view of a specific area. Detailed Implementation

[0075] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments. Example 1

[0076] Automatic registration and adaptive path planning methods for intelligent urinary catheterization robots, such as Figure 1 , Figure 2 As shown, it includes the following steps:

[0077] Acquire image data of the patient's catheter tip; in this embodiment, video or image acquisition is performed on the catheter tip of different patients, capturing multiple angles and obtaining videos (30 frames) or images in multiple states to increase the training sample size. Regarding user privacy, data is not shared between different users; users can set access permissions and passwords for their own data to protect their personal privacy.

[0078] For the collected video data, image data was extracted by taking frames one by one or at intervals. The resulting images and videos were clear and without ghosting, and the tip of the catheter tissue had good recognizability in the images.

[0079] The anterior region of the catheter tissue to be identified in the image data is labeled. In this embodiment, a labeling tool is used to label the collected data. To protect user privacy, users can choose private labeling, automatic labeling, or commissioned labeling. Private labeling refers to the user processing their own data, using a polygon tool to outline the anterior tissue and catheter port in the image, and setting the labels to 0 and 1 respectively. Automatic labeling uses a trained GLM-YOLO model to label user data, which can reduce the significant time required for manual labeling. After completion, the user needs to check and manually correct the labeled data. Commissioned labeling is mostly provided for users who are inconvenient or non-professional; users can entrust a third party to label their data. The labeling of user data needs to have a certain degree of professionalism and privacy, and different labeling methods can meet the needs of different users.

[0080] In some embodiments, the labeled user data is enhanced to a certain extent through Gaussian blur, rotation, brightness, contrast, saturation, and hue processing. Specifically, Gaussian blur is used to reduce image noise and detail; rotation is used to improve the performance of the detection network; brightness processing is used to improve the recognizability of dark or bright images; contrast processing is used to help improve image sharpness, detail, and grayscale levels; saturation processing is used to improve the color expressiveness of the image; and hue processing is used to enhance details, making the image clearer and sharper. Through data augmentation, the accuracy and generalization of the trained model are improved.

[0081] The GLM-YOLO target segmentation network (GPU accelerated) is used to detect and identify the front region of the catheter tissue in real time, and the identified region information is fed back.

[0082] like Figure 4As shown, the GLM-YOLO object segmentation network is a deep learning model that improves upon the YOLOv8 model. While retaining the core convolutional modules, C2F modules, SPFF modules, and segmentation head of the YOLOv8 model, GLM-YOLO introduces key changes to meet the requirements of pixel-level segmentation tasks while maintaining low-latency performance on mobile devices. Unlike the YOLOv8 model, GLM-YOLO employs a global-local module to replace part of the C2F module. This decision aims to address the challenges of high-resolution computation without compromising performance, while also optimizing inference speed on mobile devices. Although the number of parameters in the GLM-YOLO model is reduced, the introduction of the global-local module allows the model to better fuse semantic and spatial information, combining shallow and deep features, thereby improving model performance.

[0083] Specifically, the steps for real-time detection and identification of the anterior region of the catheter tissue using the GLM-YOLO target segmentation network include the following:

[0084] Feature extraction is performed on the output feature maps from three high-resolution spatial branches by applying 1×1 convolution and batch normalization.

[0085] Feature maps from three different resolutions are merged along the channel dimension and then used through an axial attention mechanism to obtain more comprehensive information.

[0086] The output feature map from the context branch is processed by the GLM-YOLO target segmentation network. It is processed by a combination of 1×1 convolution, batch normalization and sigmoid activation function, and then upsampled to a high resolution level by bilinear interpolation to generate semantic weights; this step helps to extract higher-level semantic information.

[0087] By using the GLM-YOLO object segmentation network, semantic weights from the context branch are multiplied element-wise with high-resolution features from the spatial branch, allowing low-level spatial features to acquire richer high-level semantic information, thereby improving the model's performance in pixel-level segmentation tasks.

[0088] The aforementioned axial attention mechanism aims to reduce the computational complexity of the original global self-attention, calculating self-attention on a single axis at a time, and stacking attention modules on a horizontal and a vertical axis to obtain the global receptive field. For example... Figure 5 As shown, the specific process is as follows:

[0089] Assumption For the input feature map, for The linear projection, i.e. , , ,and , For learning weights, These represent the height, width, and number of channels of the feature map, respectively. The global attention mechanism can be described as:

[0090] (1)

[0091] in, To indicate This refers to all locations on the feature map.

[0092] To achieve more efficient computation while aggregating global information, a more aggressive strategy is adopted: horizontal and vertical compression is achieved by averaging the query feature mappings in the horizontal and vertical directions. Based on global information, through... get ,in , To learn the weights, global semantic information is extracted from the squeeze axis attention, described as follows: (2)

[0093] in, Represents tensor Arranged according to a given dimension, Let represent a vector in which all elements are equal to 1. and Simultaneously perform the extrusion operation to obtain , The extrusion operation retains all information on a single axis, thus greatly reducing the global semantic information of equation (3), as described below: (3)

[0094] Each location in the feature map propagates information only along two compressed axial features. Since it lacks positional information from the feature map, an attention effect can be achieved through compressed axial embedding; specifically, this utilizes positional embedding. Make and We know their positions within the compression axial features. Therefore, the axial attention mechanism can be expressed as:

[0095] (4)

[0096] In this embodiment, the training set, validation set, and test set are randomly allocated in an 8:1:1 ratio to ensure that the trained model has a high recognition rate and accuracy. The training set is used for data samples for model fitting, the validation set is used to adjust the model's hyperparameters and for preliminary evaluation of the model, and the test set is used to evaluate the generalization ability of the final model.

[0097] Based on the real-time urethral orifice segmentation information fed back by the GLM-YOLO target segmentation network, the centroid of the segmented urethral orifice region is located. The centroid is then compared with the center point of the endoscopic image to determine its position. The flexible catheter endoscope is then registered with this centroid using the three-axis control system. The intelligent catheterization robot software interface is shown below. Figure 3 As shown, the structure of the intelligent catheterization robot is as follows: Figure 6 As shown.

[0098] The aforementioned flexible catheter endoscope triaxial control system has three control axes, typically including a transverse X-axis, a longitudinal Y-axis, and a depth Z-axis. These three control axes allow control of the flexible catheter endoscope's movement direction and advance depth for better observation and management of patients' urethral and bladder problems. This flexible catheter endoscope triaxial control system includes an X-axis motor, a Y-axis motor, a Z-axis motor, and a control unit; wherein,

[0099] The X-axis motor is used to control the horizontal movement of the urinary catheter;

[0100] The Y-axis motor is used to control the vertical movement of the urinary catheter;

[0101] The Z-axis motor is used to control the movement of the catheter in the forward and backward directions;

[0102] The control unit is used to receive motion commands transmitted by the system and coordinate and control the drives of the X-axis motor, Y-axis motor, and Z-axis motor.

[0103] The flexible ureteroscope is equipped with a high-resolution camera and lighting system at its tip. It is used to provide clear images of the inside of the patient's urethra and bladder, allowing doctors to clearly see the inside of the urethra and bladder. The visual system transmits the images to a display screen for the doctor's reference.

[0104] Specifically, aligning the flexible catheter endoscope with the center of mass using the three-axis control system includes the following steps:

[0105] Images acquired by a flexible catheter endoscope were analyzed using the GLM-YOLO segmentation algorithm, where the image height was [missing information]. , width is The urethral orifice segmentation results were obtained. ;

[0106] If the current frame contains a urethral orifice segmentation result Then calculate the urethral orifice segmentation result. outline Calculate the urethral orifice segmentation result. center of mass Next, determine the location of the centroid.

[0107] Specifically, using the coordinates of the image center point Establish a rectangular coordinate system with the origin. , The rectangular coordinate system divides the screen into four sections, starting from the top left corner and proceeding clockwise: the first section, the second section, the third section, and the fourth section.

[0108] Using the coordinates of the center point of the endoscopic image Establish a rectangular coordinate system with the origin and calculate the centroid. Distances to the horizontal axis, vertical axis, and origin: , , The AI ​​detection results and magnified local diagrams are as follows: Figure 7 As shown.

[0109] Determine the center of mass The specific rules for determining the location are as follows: The specification also stipulates that the three-axis motors of the flexible catheter endoscope drive the endoscope in three directions: lateral movement (X), longitudinal movement (Y), and depth movement (Z). The lateral movement (X) includes two actions: X to the left and X to the right; the longitudinal movement (Y) includes two actions: Y up and Y down; and the depth movement (Z) includes two actions: Z forward and Z backward. This is based on the center of mass. The specific drive method of the three-axis motor at the location is as follows:

[0110]

[0111] The magnitudes of the lateral motion X and the longitudinal motion Y are: and , It is a manually set threshold, meaning when the centroid... Coordinates of the image center point The distance between them is less than the threshold At this point, it is considered that the automatic registration of the target position has been completed, and the flexible catheter is then driven forward to enter the urethra.

[0112] The pressure of the urethral wall on the flexible catheter is detected in real time by a force feedback sensor. The detected pressure is used to determine whether the force applied by the flexible catheter to the urethral wall during the movement of the catheter is safe, so as to ensure that the force applied by the flexible catheter to the urethral wall during the movement of the catheter is safe.

[0113] In this embodiment, the force feedback sensor is a ring force feedback sensor, which is located at the front end of the catheter. The ring force feedback sensor is used to monitor the force on the front end of the catheter in real time, including its magnitude and direction. Specifically, the ring force feedback sensor converts the force into a digital signal in real time. Its initial pressure value when it is not under pressure is 0. Once a certain part of the ring force feedback sensor is subjected to pressure, the pressure value corresponding to the current position will increase.

[0114] The force feedback control system adjusts the force applied to the tip of the catheter based on the signal output from the ring force feedback sensor, ensuring that excessive pressure is avoided on the urethral wall when the catheter enters the urethra. Specifically, a safety threshold is set in the force feedback control system. When the signal output from the ring force feedback sensor reaches or exceeds the safety threshold, the three-axis control system of the flexible catheter endoscope automatically adjusts the position of the flexible catheter endoscope to reduce the pressure applied to the pressure points by the tip of the catheter.

[0115] Based on the assessment results, the optimal catheterization route is planned to minimize patient discomfort; specifically, the following steps are included:

[0116] After automatic registration with the target position, when the flexible catheter endoscope advances to the urethral opening, the current position is marked as the starting position;

[0117] Real-time force data, including magnitude and direction, is obtained from the ring force feedback sensor. The orientation of the tip of the flexible catheter is adjusted through the force feedback control system. While keeping the force on the tip of the flexible catheter within a safe threshold, the advancement of the flexible catheter is controlled.

[0118] An adaptive pathfinding algorithm calculates and compares all pressure values ​​on the ring-shaped force feedback sensor, and adjusts the navigation trajectory in conjunction with the force feedback control system to achieve force balance. Simultaneously, the adaptive pathfinding algorithm optimizes the trajectory to ensure a smooth and continuous catheter insertion process. When the flexible urethroscope successfully reaches the bladder position or the adaptive pathfinding algorithm detects an abnormality (such as obstruction), it automatically terminates the catheter insertion.

[0119] The system stores data corresponding to different users and records the path information of each catheterization. By comparing multiple data points, it generates automatic catheterization configuration parameters, enabling the system to assist users with automatic catheterization in the fastest, most efficient, and most comfortable way. Specifically, it includes the following steps:

[0120] Use LabVIEW's SQL Server database to determine the user;

[0121] If the user's historical information exists, the detailed technical indicators of each catheterization are processed and calculated, and automatic catheterization is initiated.

[0122] If no historical information exists, the user's name, gender, and age will be saved, and then a professional doctor will make a judgment and make fine adjustments. At the same time, detailed technical indicators of each catheterization when the user uses this product will be recorded, including the size of the urethral opening, the action commands of the three-axis control system of the urethral catheter after the flexible catheter is inserted into the urethra, the real-time data of the force feedback sensor, and the total time taken for the catheter to reach the target position.

[0123] The above-mentioned steps for activating automatic catheterization include the following:

[0124] Based on the established database, find all the information of the current user, integrate and compare the comprehensive data of each catheterization process, and filter out the normal catheterization process;

[0125] By comparing and analyzing the catheterization path and real-time data from the ring force feedback sensor during each catheterization, the optimal catheterization path is calculated. This path summarizes the data indicators from each previous successful catheterization, which can reduce user discomfort during use. With the increase in the number of uses, the next automatic catheterization process will be smoother.

[0126] Find the corresponding user's catheterization path information, extract the user's catheterization path information, and automatically perform catheterization.

[0127] This invention addresses the shortcomings of current intelligent catheterization robots, such as high operational expertise requirements, significant psychological burden on patients, low intelligence levels, and lack of integration with daily life. It aims to develop an easy-to-operate, intelligent, and user-friendly intelligent catheterization robot with automatic registration and adaptive path planning methods and devices. Utilizing artificial intelligence technology, the system guides the flexible catheteroscope precisely to the catheterization port. While avoiding privacy concerns, the system effectively reduces operator fatigue and repetitive tasks, while significantly improving catheterization efficiency through increased intelligence and precision. Example 2

[0128] The intelligent catheterization robot's automatic registration and adaptive path planning device implements the above-described method. For a detailed description of the method, please refer to the corresponding description in the above method embodiments, which will not be repeated here. The device includes an image acquisition module, an image processing module, a GLM-YOLO target segmentation network module, a flexible catheter endoscope three-axis control system, an automatic target position registration module, a force feedback control system, an adaptive pathfinding algorithm module, and a database management and automatic catheterization module; wherein...

[0129] The image acquisition module is used to acquire image data of the front end of the patient's urinary catheter tissue;

[0130] The image processing module is used to annotate the anterior region of the catheter tissue to be identified in the image data;

[0131] The GLM-YOLO target segmentation network module is used to detect and identify the front region of the catheter tissue in real time and provide feedback on the identified region information.

[0132] The three-axis control system of the flexible catheter endoscope has three control axes, including the transverse X-axis, the longitudinal Y-axis, and the depth Z-axis. The control axes are used to control the movement direction and advance depth of the flexible catheter endoscope.

[0133] The automatic registration module for the bullseye position is used to find the centroid of the urethral orifice segmentation region based on the real-time feedback of the urethral orifice segmentation information from the GLM-YOLO target segmentation network. The centroid of the urethral orifice segmentation region is compared with the center point of the endoscope image to determine the position of the centroid. The flexible catheter endoscope is then registered with the position of the centroid through the three-axis control system of the flexible catheter endoscope.

[0134] The force feedback control system is used to detect the pressure of the urethral wall on the flexible catheter in real time through a force feedback sensor, and to determine whether the force applied by the flexible catheter to the urethral wall during the movement of the catheter is safe.

[0135] The adaptive pathfinding algorithm module is used to plan the optimal catheter path based on the judgment results;

[0136] The database management and automatic catheterization module is used to store data corresponding to different users, and record the path information of each catheterization. By comparing multiple data, automatic catheterization configuration parameters are generated.

[0137] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0138] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0139] The above are merely embodiments of this specification and are not intended to limit the scope of one or more embodiments of this specification. Various modifications and variations can be made to one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of one or more embodiments of this specification.

Claims

1. An automatic registration and adaptive path planning device for an intelligent urinary catheterization robot, comprising an automatic registration and adaptive path planning method for an intelligent urinary catheterization robot, characterized in that: It includes an image acquisition module, an image processing module, a GLM-YOLO target segmentation network module, a three-axis control system for flexible catheter endoscopes, an automatic target registration module, a force feedback control system, an adaptive pathfinding algorithm module, and a database management and automatic catheterization module; among which, The image acquisition module is used to acquire image data of the front end of the patient's urinary catheter tissue; The image processing module is used to annotate the anterior region of the catheter tissue to be identified in the image data; The GLM-YOLO target segmentation network module is used to detect and identify the anterior region of the catheter tissue in real time and provide feedback on the identified region information. Specifically, it includes: extracting features from the output feature maps of three high-resolution spatial branches; merging the feature maps from the three different resolutions along the channel dimension and obtaining more comprehensive information through an axial attention mechanism; processing the output feature maps from the context branch through the GLM-YOLO target segmentation network, and then upsampling them to a high-resolution level through bilinear interpolation to generate semantic weights; and multiplying the semantic weights from the context branch with the high-resolution features from the spatial branch element-wise through the GLM-YOLO target segmentation network, enabling the low-level spatial features to obtain richer high-level semantic information. The three-axis control system of the flexible catheter endoscope has three control axes, including the horizontal X-axis, the vertical Y-axis and the depth Z-axis. The control axes are used to control the movement direction and advance depth of the flexible catheter endoscope. The automatic registration module for the target position is used to find the centroid of the urethral orifice segmentation region based on the real-time feedback of the urethral orifice segmentation information from the GLM-YOLO target segmentation network. The centroid of the urethral orifice segmentation region is compared with the center point of the endoscope image to determine the position of the centroid. The flexible catheter endoscope is then registered with the position of the centroid through the three-axis control system of the flexible catheter endoscope. The force feedback control system is used to detect the pressure of the urethral wall on the flexible catheter in real time through a force feedback sensor, and to determine whether the force applied by the flexible catheter to the urethral wall during the movement of the catheter is safe by using the detected pressure. The adaptive pathfinding algorithm module is used to plan the optimal catheter path based on the judgment result; The database management and automatic catheterization module is used to store data corresponding to different users, and record the path information of each catheterization. By comparing multiple data, automatic catheterization configuration parameters are generated.

2. The intelligent catheterization robot automatic registration and adaptive path planning device as described in claim 1, characterized in that: The image processing module is further configured to enhance the labeled user data by applying Gaussian blur, rotation, brightness, contrast, saturation, and hue processing after the step of annotating the anterior contour of the urinary catheter tissue and the urinary catheter opening in the image data.

3. The intelligent catheterization robot automatic registration and adaptive path planning device as described in claim 1, characterized in that: The axial attention mechanism is expressed as follows: in, , Represents tensor Arranged according to a given dimension, Represents a vector in which all elements are equal to 1, positional embedding. , for linear projection, , , , Given the input feature map, and , For learning weights, These represent the height, width, and number of channels of the feature map, respectively.

4. The intelligent catheterization robot automatic registration and adaptive path planning device as described in claim 1, characterized in that: The three-axis control system for the flexible catheter endoscope includes an X-axis motor, a Y-axis motor, a Z-axis motor, and a control unit; wherein, The X-axis motor is used to control the horizontal movement of the urinary catheter; The Y-axis motor is used to control the vertical movement of the urinary catheter; The Z-axis motor is used to control the movement of the catheter in the forward and backward directions; The control unit is used to receive motion commands transmitted by the system and coordinate and control the drives of the X-axis motor, Y-axis motor, and Z-axis motor.

5. The intelligent catheterization robot automatic registration and adaptive path planning device as described in claim 4, characterized in that: The flexible catheter endoscope is equipped with a high-resolution camera and an illumination system at its tip, and is used to provide clear imaging of the patient's urethra and bladder. The process of aligning the flexible catheter endoscope with the center of mass using the three-axis control system of the flexible catheter endoscope includes the following steps: The urethral orifice segmentation result is obtained by detecting the image acquired by the flexible catheter endoscope using the GLM-YOLO segmentation algorithm. If a urethral opening segmentation result exists in the current frame, calculate the contour of the urethral opening segmentation result, calculate the centroid of the urethral opening segmentation result, and determine the position of the centroid.

6. The intelligent catheterization robot automatic registration and adaptive path planning device as described in claim 5, characterized in that: The process of aligning the flexible catheter endoscope with the center of mass using the three-axis control system of the flexible catheter endoscope includes the following steps: Using the coordinates of the image center point Establish a rectangular coordinate system with the origin. This system divides the image into four intervals, starting from the top left corner and proceeding clockwise: interval one, interval two, interval three, and interval four. , , Image height, Image width; Using the coordinates of the center point of the endoscopic image Establish a rectangular coordinate system with the origin and calculate the centroid. Distances to the horizontal axis, vertical axis, and origin: , , ; Determine the center of mass The specific rules for determining the location are as follows: The specification also stipulates that the three-axis motors of the flexible catheter endoscope drive the endoscope in three directions: lateral movement (X), longitudinal movement (Y), and depth movement (Z). The lateral movement (X) includes two actions: X to the left and X to the right; the longitudinal movement (Y) includes two actions: Y up and Y down; and the depth movement (Z) includes two actions: Z forward and Z backward. This is based on the center of mass. The specific drive method of the three-axis motor at the location is as follows: The magnitudes of the lateral motion X and the longitudinal motion Y are: and , It is a manually set threshold, when the centroid Coordinates of the image center point The distance between them is less than the threshold At this point, it is considered that the automatic registration of the target position has been completed, and the flexible catheter is then driven forward to enter the urethra.

7. The intelligent catheterization robot automatic registration and adaptive path planning device as described in claim 1, characterized in that: The force feedback sensor is a ring force feedback sensor, which is located at the front end of the catheter. The ring force feedback sensor is used to monitor the force on the front end of the catheter in real time, including its magnitude and direction. The step of determining whether the force applied to the urethral wall by the flexible catheter endoscope during its movement is safe by detecting pressure includes the following steps: When the signal output by the ring force feedback sensor reaches or exceeds the safety threshold, adjust the position of the flexible catheter endoscope to reduce the pressure exerted by the catheter tip on the pressure point.

8. The intelligent catheterization robot automatic registration and adaptive path planning device as described in claim 7, characterized in that: The step of planning the optimal catheter route based on the judgment result includes the following steps: After automatic registration with the target position, when the flexible catheter endoscope advances to the urethral opening, the current position is marked as the starting position; By adjusting the position of the tip of the flexible catheter endoscope, the forward movement of the flexible catheter endoscope is controlled while keeping the force on the tip of the flexible catheter endoscope within a safe threshold. The adaptive pathfinding algorithm calculates and compares all pressure values ​​on the ring force feedback sensor, and adjusts the navigation trajectory in conjunction with the force feedback control system to achieve force balance. At the same time, the adaptive pathfinding algorithm optimizes the trajectory to ensure that the catheter insertion process is smooth and continuous. The catheter insertion is terminated when the urethral endoscope is successfully positioned in the bladder or when the adaptive pathfinding algorithm detects an abnormality.

9. The intelligent catheterization robot automatic registration and adaptive path planning device as described in claim 1, characterized in that: The process of storing data corresponding to different users, recording the path information of each catheterization, and generating automatic catheterization configuration parameters through multiple data comparisons includes the following steps: Use a database to determine the user; If the user's historical information exists, the detailed technical indicators of each catheterization are processed and calculated, and automatic catheterization is initiated. If no historical information exists, the user's name, gender, and age will be saved. At the same time, detailed technical indicators of each catheterization will be recorded, including the size of the urethral opening, the action commands of the three-axis control system of the flexible catheter after it enters the urethra, the real-time data of the force feedback sensor, and the total time taken for the catheter to reach the target position.

10. The intelligent catheterization robot automatic registration and adaptive path planning device as described in claim 9, characterized in that: The automatic catheterization activation process includes the following steps: Based on the established database, find all the information of the current user, integrate and compare the comprehensive data of each catheterization process, and filter out the normal catheterization process; By comparing and analyzing the catheterization path and real-time data from the force feedback sensor during each catheterization, the optimal catheterization path is calculated. Automatic catheterization is performed based on the optimal catheterization path.

Images