A snake-shaped bionic catheterization robot and its control method
Through the snake-shaped bionic catheter robot, it uses Markov decision-making process and reinforcement learning optimization control strategy, and combines deformation memory alloys and micro sensors to achieve active bending and tracing of the catheter, solving the problems of catheter damage and operation difficulty. It improves the convenience and accuracy of catheterization and helps patients recover their autonomous urination function.
Patent Information
- Application Number
- CN202410664377.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-27
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-05-27
Smart Images

Figure CN118593875B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field related to urinary catheters, and in particular relates to a snake-shaped bionic urinary catheterization robot and a control method thereof. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] Catheterization is a commonly used technical means in clinical practice. It involves inserting a special catheter through the urethra into the bladder to drain urine. Its purpose is to relieve urine retention caused by various reasons. It is often used for patients with spontaneous urination disorders.
[0004] In clinical practice, regular intermittent catheterization can help patients exercise and restore autonomous urination function, avoid problems such as excessive bladder pressure and bladder-ureteral reflux caused by long-term urine retention, and effectively protect kidney function. The catheterization process is usually that the operator (the operator himself or others) inserts a sterile and lubricated catheter into the patient's urethra, and then slowly pushes the catheter until urine flows out of the catheter. However, this method has obvious disadvantages. The movement and turning of the catheter in the urethra are completely dependent on the external force applied. The external force forces the catheter to undergo passive deformation, which can easily damage the urethra and cause urinary tract infection. Especially when patients at home perform autonomous urination operations, the lack of experience and illness lead to impaired motor and sensory functions, making the operation more difficult and causing great inconvenience to the patients. In addition, most of the patients with catheterization are elderly people, and the grasp of catheterization time and the recording of catheterization conditions are often not accurate enough. This unscientific catheterization operation is often not conducive to the recovery of patients' physical functions.
[0005] Therefore, how to provide a urinary catheter to solve the above-mentioned problems such as inconvenient operation and damage to the urethra is a problem that needs to be solved at present. Summary of the invention
[0006] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a snake-shaped bionic catheterization robot and a control method thereof, which can realize active tracking of the catheterization robot during insertion, thereby avoiding friction damage to the urethra caused by the catheter to a great extent.
[0007] To achieve the above object, a first aspect of the present invention provides a control method for a serpentine bionic urinary catheterization robot, which is applied to the serpentine bionic urinary catheterization robot. The control method comprises:
[0008] According to the pressure and bending degree of the snake-like bionic catheterization robot during its movement, the motion control problem of the snake-like bionic catheterization robot is described as a Markov decision process.
[0009] According to the Markov decision process, reinforcement learning is used to perform training optimization to obtain the optimal control strategy;
[0010] The pressure on the serpentine bionic catheterization robot during its current motion is obtained and input into the optimal control strategy to obtain a control execution action for the current motion bending degree of the serpentine bionic catheterization robot.
[0011] The second aspect of the present invention provides a snake-like bionic catheterization robot, comprising: a control box, a flexible part, and an active bending part arranged at one end of the flexible part; the active bending part comprises an elastic member, and a deformation memory alloy arranged around the elastic member; the control box is used to control the movement bending degree of the deformation memory alloy according to the above-mentioned control method.
[0012] A third aspect of the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, executes a control method for a snake-like bionic urinary catheterization robot.
[0013] One or more of the above technical solutions have the following beneficial effects:
[0014] In the present invention, the motion control problem of the serpentine bionic catheterization robot is described as a Markov decision process, and reinforcement learning is used for optimization to obtain the optimal strategy. The pressure on the serpentine bionic catheterization robot during the current motion is input into the optimal strategy to obtain the control execution action of the motion bending degree. By controlling and adjusting the motion bending degree based on the pressure obtained in real time, the catheterization robot can actively track during insertion, thus greatly avoiding the friction damage of the catheter to the urethra.
[0015] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0017] Figure 1 This is a schematic diagram of the structure of the snake-shaped bionic catheterization robot in the first embodiment of the present invention;
[0018] Figure 2 This is a schematic diagram of the design principle of the active bending part in the first embodiment of the present invention;
[0019] Figure 3 This is a schematic diagram of the three-layer structure design of the flexible hose and the sensor installation in the first embodiment of the present invention;
[0020] Figure 4 This is a schematic diagram of the active tracking of the catheterization robot and a schematic diagram of the control box design in the first embodiment of the present invention;
[0021] Figure 5 The movement principle of the snake-like bionic catheterization robot in the second embodiment of the present invention;
[0022] Figure 6 Schematic diagram of two training stages of reinforcement learning for motion control of a urinary catheterization robot in the second embodiment of the present invention;
[0023] Figure 7 This is a schematic diagram of the intelligent urinary catheterization system in the second embodiment of the present invention;
[0024] Figure 8 This is a schematic diagram of a test sample of the urinary catheterization robot in the second embodiment of the present invention;
[0025] In the figure, 1. control box, 2. flexible part, 3. active bending part, 4. positive electrode, 5. nickel-titanium alloy wire, 6. ultra-fine steel wire spring, 7. negative electrode, 8. micro pressure sensor, 9. micro camera, 10. hydrophilic coating, 11. silicone material, 12. silicone material + hydrophilic coating, 13. simulated urethra, 101. controller, 102. drainage hole, 103. direction control joystick, 104. ultra-fine power harness, 105. ring wire. DETAILED DESCRIPTION
[0026] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.
[0027] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.
[0028] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.
[0029] Embodiment 1
[0030] like Figure 1 As shown, this embodiment discloses a snake-like bionic catheterization robot, comprising: a control box 1, a flexible part 2, and an active bending part 3 arranged at one end of the flexible part 2; the active bending part 3 comprises an elastic member, and a deformation memory alloy arranged around the elastic member.
[0031] The flexible part 2 is specifically a flexible hose.
[0032] In terms of overall functionality: the control box 1 is mainly responsible for driving and controlling the active bending part 3; the flexible hose is a pipe that guides the discharge of urine, and it does not have the ability to actively deform itself, and can only undergo passive deformation under the action of external force; the active bending part 3 can be regarded as an artificial muscle wrapped at the end of the flexible hose, which can bend to varying degrees under stimulation of voltages of different intensities.
[0033] Depending on the voltage signal control of the control box 1, the active bending portion 3 can be bent in different degrees and directions, thereby guiding the flexible hose to move in the urethra.
[0034] The active bending part 3 of this embodiment is the core component for realizing active tracking of the urinary catheter, and its principle is to imitate the muscle control during the movement of a snake.
[0035] like Figure 2 As shown, the entire active bending portion 3 is composed of an extremely fine steel wire spring 6 and six nickel-titanium alloy wires 5 evenly fixed around it. Nickel-titanium alloy is a shape memory alloy with excellent performance and is widely used in aerospace, industry, medical and other fields.
[0036] The motion control scheme designed in this embodiment is to achieve bending of different degrees and directions by controlling the voltage stimulation of the six nickel-titanium alloy wires 5; by controlling the degree of movement and bending of the six nickel-titanium alloy wires, active bending and tracking of any angle of the end of the catheter can be achieved, thereby guiding the catheter to move smoothly in the urethra and avoiding damage to the urethra.
[0037] It can be understood that, on the outside of the ultra-fine steel wire spring 6, 6 nickel-titanium alloy wires 5 are arranged at equal intervals along the circumferential direction, one end of the nickel-titanium alloy wire 5 is fixed to the upper end of the ultra-fine steel wire spring 6, and the other end of the nickel-titanium alloy wire 5 is fixed to the lower end of the ultra-fine steel wire spring 6. The nickel-titanium alloy wire 5 has the characteristic of shrinking in length after voltage is applied, and the greater the voltage applied, the greater the shrinkage, and the length returns to normal after power is cut off. By utilizing this characteristic of the nickel-titanium alloy wire 5, the ultra-fine steel wire spring 6 is bent and deformed.
[0038] The flexible hose of this embodiment adopts a three-layer structure design, and its layered structure is as follows: Figure 3 As shown. The innermost and outermost layers are made of silicone material, which is soft, elastic, and has good biocompatibility and corrosion resistance. A hydrophilic coating is also attached to the surface of the outermost silicone material. When wet, the functional coating is activated by water to form a colorless and transparent hydrogel. The hydrogel coating has a high lubricity and can withstand repeated friction.
[0039] A layer of extremely fine metal mesh made of stainless steel is embedded between the two layers of silicone material, which ensures the toughness and strength of the catheter while taking into account flexibility. At the end of the flexible hose, the active bending part is embedded between the silicone material and the metal mesh, which can ensure that the active bending layer is firmly fixed on the one hand, and facilitate the layout of wires, transmission lines, etc. on the other hand; finally, the end of the flexible hose is covered with a layer of hydrophilic coating, thereby sealing the three-layer tubular structure.
[0040] In addition, in this embodiment, a miniature camera 9 and a miniature pressure sensor 8 are installed at the end of the flexible hose to collect visual data and intra-bladder pressure data; with the help of visual data and intelligent recognition algorithms, automatic recognition and positioning of the catheterization port and automatic guidance of the catheterization path can be achieved; with the help of bladder pressure data, the bladder urination situation can be effectively detected to realize intelligent urination.
[0041] like Figure 4 As shown, the serpentine bionic active tracking catheterization robot designed in this embodiment can autonomously track in the curved urethral structure and guide the catheter to move in the urethra. Through the handheld control box, the operator can flexibly control the steering of the active bending layer. Specifically, the control box is equipped with an APM32 controller for driving and controlling the active bending layer. The six nickel-titanium alloy wires in the active bending part are connected to the controller through ultra-fine wires. A direction control joystick is integrated on the control box, through which the steering of the active bending layer can be flexibly controlled, thereby guiding the flexible hose to move in different directions. A drainage hole is reserved at the tail of the control box, which can be connected to a device such as a urine storage box to store the drained urine.
[0042] Embodiment 2
[0043] This embodiment provides a method for a serpentine bionic catheterization robot, which can be used to control the serpentine bionic catheterization robot as described in the first embodiment. The control method includes:
[0044] According to the pressure and bending degree of the snake-like bionic catheterization robot during its movement, the motion control problem of the snake-like bionic catheterization robot is described as a Markov decision process.
[0045] According to the Markov decision process, reinforcement learning is used to perform training optimization to obtain the optimal control strategy;
[0046] The pressure on the serpentine bionic catheterization robot during its current motion is obtained and input into the optimal control strategy to obtain a control execution action for the current motion bending degree of the serpentine bionic catheterization robot.
[0047] Six micro pressure sensors are integrated at the ends of the six alloy wires. When the catheter travels in the urethra, the pressure sensors will provide real-time feedback of the resistance values from different directions during the travel process. Based on this, the deformation direction and degree of the active bending layer can be adjusted to achieve active tracking when the catheter is inserted. Therefore, how to adjust the control voltage of the six alloy wires according to the values fed back by the pressure sensors to achieve the desired bending direction and degree is a problem to be solved by the motion control of the snake-like bionic robot in this embodiment.
[0048] This embodiment proposes a motion control algorithm for a catheterization robot based on reinforcement learning. Specifically, the present invention constructs a 1:1 restoration of a real snake-like bionic catheterization robot in the CoppeliaSim simulation platform. The key components are 6 shape memory alloy wires and 6 pressure sensors. This embodiment simplifies the shape memory alloy wire into a deformable part in the simulation platform and configures a linear contraction dynamics model for it.
[0049] The specific details of the reinforcement learning algorithm for the motion control of the catheterization robot are as follows:
[0050] Problem description: Figure 2 As shown, the motion control problem of the catheterization robot in the present invention can be described as, given a set of pressure sensor values F = {f1, f2, f3, f4, f5, f6}, solving the control voltage values V = {v1, v2, v3, v4, v5, v6} of the six nickel-titanium alloy wires, so that the active bending part at the end of the catheter can achieve a suitable bending angle and degree. The modeling process can be expressed as V = φ(F).
[0051] To fit the model, the problem is defined as a Markov decision process (MDP). At each time step t, the robot performs action a t , according to the state transition probability distribution P, the current state s t Go to next state s t+1 , and then get an immediate reward r t .
[0052] The goal of reinforcement learning is to obtain an optimal strategy π through training, so as to maximize the expected reward under the action of the discount factor γ, which is defined as:
[0053]
[0054] To solve this optimization problem, this embodiment adopts the Proximal Policy Optimization (PPO) algorithm framework. The specific training process is as follows: In the constructed simulation platform, randomly generate training scenarios for policy learning training. In each training round, the catheterization robot operates in a randomly generated environment. During this process, the value of the pressure sensor and the length of the alloy wire at each time step t are recorded to obtain the current state s t . Set the current state s t Input to the policy network π(θ) to obtain the action output a t . Catheterization robot performs action a t , the reward function value r can be calculated according to the set reward function t According to the reward function value, the policy network is updated to obtain a new policy π′(θ). In the training environment, the above training process is repeated continuously, and the policy network is continuously updated until the policy network converges, and the optimal control strategy can be obtained.
[0055] State-action space: The state space consists of the sensor pressure value and the current length of the alloy wire, expressed as S = {F, L}, where F = {f1, f2, f3, f4, f5, f6}, L = {l1, l2, l3, l4, l5, l6}, representing the values collected by the pressure sensors in 6 different directions and the current lengths of the 6 nickel-titanium alloy wires. The pressure value is in N, and the length value is in mm.
[0056] The action space is defined as the control voltage value V = {v1, v2, v3, v4, v5, v6} of the six nickel-titanium alloy wires, where the voltage applied to each alloy wire ranges from 2V to 6V.
[0057] Reward function: In order to encourage the snake-like bionic robot to achieve the expected deformation angle and degree, a reward function R(s t ,a t ).like Figure 3 As shown in , the entire learning process is divided into two stages, the first stage is static training, and the second stage is dynamic training. Figure 7 As shown in the static training stage, the target point p is randomly set within the reachable range of the catheterization robot. target =(x1,y1,z1), records the actual arrival position p of the end of the current catheterization robot reach =(x2,y2,z2). At this time, the target point and the actual arrival point will form different angles with the horizontal axis (X or Y axis). The angle difference between the two is recorded as Reward function R during static training stage1 The definition is as follows:
[0058]
[0059] In the dynamic training phase, 60 urethra models with different structures were constructed in the simulation environment. In each training round, one model was randomly selected and the placement posture was randomly set. Then the catheterization robot was allowed to explore autonomously in the simulated urethra. During the exploration, the values of the six force sensors F = {f1, f2, f3, f4, f5, f6} were recorded. In theory, when the direction of travel of the end of the catheterization robot is inconsistent with the urethra structure, obvious friction will occur, and the force sensor value will be larger at this time.
[0060] Reward function R during dynamic training stage2 Defined as:
[0061]
[0062] Among them, α, β, and λ are hyperparameters, which are set to 0.1, 1.2, and 0.1 respectively in this embodiment.
[0063] Clinically, female patients cannot see the location of the urethral opening when conducting autonomous urination due to the special position of the urethra, making it difficult to accurately find the insertion position of the catheter. In the present invention, by installing a miniature camera at the end of the catheter, the visual field in front of the catheter can be captured in real time, and displayed in real time through a mobile phone or an external display device, which can help patients insert the catheter more accurately. On this basis, the present invention designs a urethral opening recognition and positioning algorithm, which can accurately locate the urethral opening in the video image, and can realize insertion path guidance in combination with the camera's own calibration parameters, greatly reducing the difficulty of catheter insertion in female patients.
[0064] Specifically, the present invention constructs a urethral orifice recognition and positioning model based on YOLOv8, the details of which are as follows:
[0065] Model input and output: The present invention defines urethral opening identification and positioning as an image detection task. Given an image I as input, the detection network will perform feature extraction and result prediction, and finally output B = (x, y, w, h, confidence), which represents the rectangular box where the urethral opening is located in the image, where (x, y) is the image coordinate of the upper left corner of the rectangular box, (w, h) is the width and height of the rectangular box, and confidence is the prediction confidence of the rectangular box.
[0066] Loss function calculation: The loss function provides optimization signals during the network training process, including positive and negative sample allocation and loss value calculation. The positive and negative sample allocation strategy is to assign different weights to each training sample, so that the model pays more attention to important samples and improves the training effect. The present invention adopts the Task-Aligned Assigner dynamic allocation strategy to dynamically adjust the proportion of positive and negative samples during the training process. The loss function value includes classification loss and regression loss, where the classification loss calculation adopts the VFL loss function (Varifocal Loss) and the regression loss calculation adopts the DFL loss function (Distribution Focal Loss), which are defined as formulas (4) and (5) respectively.
[0067]
[0068] DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 )) (5)
[0069] In formula (4), q is the intersection-over-intersection ratio of the predicted rectangle and the true label rectangle. q>0 means that the two boxes intersect, representing a positive sample; q=0 means that the two boxes do not intersect, representing a negative sample. p represents the predicted probability, and α and γ are hyperparameters. In formula (5), y is the predicted target, y i and i+1 are the two labels closest to y,
[0070] like Figure 8 As shown, this embodiment collects real-time data through visual sensors and pressure sensors, and designs an intelligent catheterization APP in combination with intelligent analysis algorithms to achieve intelligent monitoring of catheterization conditions. Figure 6As shown in the figure, the smart catheterization APP obtains sensor data and calls cloud services through wireless transmission. The APP contains functional modules such as "obtaining real-time video", "urethral opening detection", "obtaining pressure data", "viewing urination records", and "intelligent catheterization reminder". After the real-time video acquisition function is turned on, the video data obtained by the visual sensor will be displayed in real time on the mobile phone, which is convenient for users to view. The urethral opening recognition and positioning algorithm is deployed in the cloud service. The mobile phone can call the urethral opening detection service in the cloud and display the detection results on the mobile phone. Each time the catheterization operation is performed, the pressure sensor will record the changes in the bladder pressure, and the relevant data will be uploaded and saved in the cloud. Users can also view the urination records on the mobile phone, and the APP will show the detailed information and analysis of each urination to the user. The APP also has an intelligent reminder function. On the one hand, when the pressure sensor detects that the pressure in the bladder is lower than a certain value during urination, the APP will remind the user to complete urination and remove the catheter in time; on the other hand, the cloud data analysis algorithm analyzes the past urination records and can generate a scientific urination plan suitable for the user in a targeted manner. The APP automatically sets an alarm to remind the user to urinate at the correct time.
[0071] The snake-like bionic active tracking catheterization robot designed in this embodiment was tested in a tracking task. In the test, a total of 100 test samples were constructed. Figure 5 As shown, in each test sample, the operation area is a 30cm×20cm workspace, in which several target position points are randomly set, and the catheterization robot must pass through the set target points in sequence. If all target points are reached successfully, the test is considered successful; if there are missed target points, the test is considered to have failed. In the end, the serpentine bionic active tracking catheterization robot designed in this embodiment achieved a 96% mission success rate, which fully verified the effectiveness of the catheterization robot. By analyzing the failure cases, it was found that the position span of the two front and rear target points is usually large, resulting in a sharp turn. In actual clinical applications, the patient's urethral structure usually does not have a large degree of curvature, so the catheterization machine designed in this embodiment can fully meet the clinical use needs.
[0072] The snake-shaped bionic active tracking catheterization robot designed in this embodiment overcomes the backward method of traditional catheterization that relies on external force to passively deform the catheter. By imitating the muscle control movement mechanism of the snake and through the design and control of the active bending part, the convenience and flexibility of inserting the catheter into the urethra are improved, the damage to the urethra is greatly reduced, and urinary tract infection is avoided.
[0073] The catheterization robot motion control algorithm based on reinforcement learning in this embodiment can adjust the deformation direction and degree of the active bending layer according to the pressure sensor feedback, thereby achieving active tracking when the catheter is inserted, and greatly avoiding friction damage to the urethra caused by the catheter.
[0074] The urethral opening recognition and positioning algorithm designed in this embodiment can accurately locate the urethral opening in the image. Further combined with the camera's own calibration parameters, it can realize intelligent guidance of the catheter insertion position, greatly reducing the difficulty of catheter insertion in female patients and avoiding urethral injury and urinary tract infection.
[0075] This embodiment collects real-time data through visual sensors and pressure sensors, and designs an intelligent catheterization APP in combination with intelligent analysis algorithms, which realizes intelligent monitoring of catheterization conditions and can generate scientific urination plans suitable for users in a targeted manner to help patients recover as soon as possible.
[0076] Embodiment 3
[0077] The purpose of this embodiment is to provide a storage medium.
[0078] A storage medium stores a computer program, which executes the steps of the above method when executed by a processor.
[0079] The steps involved in the above embodiment 3 correspond to the method embodiment 2, and the specific implementation method can refer to the relevant description part of embodiment 2. The term "storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.
[0080] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.
[0081] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.
Claims
1. A control method for a snake-like bionic catheterization robot, applied to the snake-like bionic catheterization robot, It is characterized in that The control method comprises: According to the pressure and bending degree of the snake-like bionic catheterization robot during its movement, the motion control problem of the snake-like bionic catheterization robot is described as a Markov decision process. According to the Markov decision process, reinforcement learning is used for training optimization to obtain the optimal control strategy; in the training stage, static training and dynamic training are included; the static training is: randomly setting a target point within the reachable range of the end of the serpentine bionic catheterization robot, and constructing a first reward function based on the angle formed by the actual position reached by the end of the serpentine bionic catheterization robot and the target point; the dynamic training is: constructing different urethra models, and constructing a second reward function based on the pressure value of the serpentine bionic catheterization robot during its autonomous movement in the urethra model; the Markov decision process is trained based on the first reward function and the second reward function; The first reward function is: The target point coordinates are (x 1 ,y 1 ,z 1 ), the coordinates of the actual position reached by the end of the snake-shaped bionic catheterization robot are (x 2 ,y 2 ,z 2 ), α, β are hyper parameters, and θ is the angle between the target point and the actual arrival point and the X or Y axis; The second reward function is: Among them, f i is the pressure value, λ is a hyperparameter; The pressure on the serpentine bionic catheterization robot during its current motion is obtained and input into the optimal control strategy to obtain a control execution action for the current motion bending degree of the serpentine bionic catheterization robot.
2. A control method for a snake-like bionic catheterization robot as claimed in claim 1, It is characterized in that Also includes: The current moving front image of the serpentine bionic catheterization robot is obtained, and the urethral opening recognition and positioning model is used to recognize the current moving front image of the serpentine bionic catheterization robot to obtain the position of the urethral opening; and the moving direction of the serpentine bionic catheterization robot is controlled according to the obtained urethral opening position.
3. The control method of the snake-like bionic catheterization robot according to claim 1, It is characterized in that During the training process, the pressure value and bending length measured at each time step are used as the current state, and the current state is input into the policy network to obtain the output action; the reward function value is calculated according to the first reward function or the second reward function; the policy network is updated according to the calculated reward function value, and training iterations are continuously performed until the policy network converges to obtain the optimal control strategy.
4. A snake-like bionic catheterization robot, It is characterized in that include: A control box, a flexible portion, and an active bending portion disposed at one end of the flexible portion; The active bending part includes an elastic member and a deformation memory alloy arranged around the elastic member; The control box is used to control the movement bending degree of the deformation memory alloy according to the control method according to any one of claims 1-3.
5. A snake-like bionic catheterization robot as claimed in claim 4, It is characterized in that The outermost layer of the flexible part is wrapped with a hydrophilic coating.
6. A snake-like bionic urinary catheterization robot as claimed in claim 4, It is characterized in that The deformation memory alloy is nickel-titanium alloy wire.
7. A storage medium, It is characterized in that Used to store computer instructions, which, when executed by a processor, implement the various steps of the control method of the snake-like bionic catheterization robot as described in any one of claims 1 to 3.
Citation Information
Patent Citations
Automatic registration and adaptive path planning method and device for intelligent urinary catheterization robot
CN117860985A
System and control procedure of a robot for interventional surgery and medium
DE112022000050T5
Learning to schedule control fragments for physics-based character simulation and robots using deep q-learning
US20180089553A1
Intelligent medical image landmark detection
US9569736B1
