Medical robot control system based on deep reinforcement learning algorithm

Abstract

The invention provides a medical robot control system based on a deep reinforcement learning algorithm, and relates to the technical field of medical robots. Comprising a magnetic control module of the magnetic control type miniaturized medical robot, a robot mechanical arm module, a robot mechanical arm kinematics analysis and control system design, mechanical arm control system design and construction, and a communication system and man-machine interaction operation design. According to the medical robot control system based on the deep reinforcement learning algorithm, the medical robot can autonomously learn how to execute tasks through interaction with the environment, detailed program design does not need to be carried out in advance, the medical robot can process a large amount of high-dimensional medical data, and the efficiency is improved. The medical robot control system based on deep reinforcement learning has high adaptability and generalization ability when facing a complex medical environment, can help the robot to realize interaction and communication with human doctors, patients and other robots, improves the intelligence and adaptability of the robot, and is high in automation and intelligence degree.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of medical robots, and specifically to a medical robot control system based on a deep reinforcement learning algorithm. Background Art

[0002] Magnetically controlled miniaturized medical robots can be controlled remotely by external magnetic fields, actively move in narrow cavities of the human body, and inspect and treat deep lesions in the human body. They have great potential in the field of clinical medicine. According to different scales, magnetically controlled miniaturized medical robots can be divided into centimeter-millimeter scale capsule robots, millimeter-micrometer scale continuum robots, and micrometer-nanometer scale micro robots. The magnetic control principles, structural designs, and potential medical applications of each are systematically summarized, and the latest research progress is reviewed. The current magnetically controlled miniaturized medical robots have problems such as single medical functions, lack of automation and intelligence. For this reason, this application now proposes a medical robot control system based on a deep reinforcement learning algorithm. Summary of the invention

[0003] The present invention mainly provides a medical robot control system based on a deep reinforcement learning algorithm to solve the technical problems raised in the above background technology.

[0004] The technical solution adopted by the present invention to solve the above technical problems is:

[0005] A medical robot control system based on a deep reinforcement learning algorithm includes a magnetic control module of a magnetically controlled miniaturized medical robot, a robot mechanical arm module, a robot mechanical arm kinematic analysis and control system design, a mechanical arm control system design and construction, and a communication system and human-computer interaction operation design. The control system design based on deep reinforcement learning includes three links: perception, decision-making, and execution. The mechanical arm control system design and construction include controller design and hardware construction. The communication system and human-computer interaction operation design include communication system design and human-computer interaction operation design. The specific steps for implementing the robot control system are as follows:

[0006] S1, the magnetic control module interacts with the magnetic material inside the robot through the external magnetic field to control the movement trajectory and speed of the robot;

[0007] S2, the robotic arm module is responsible for performing specific medical tasks;

[0008] S3. Establish a kinematic model of the robot by analyzing the kinematic characteristics of the robot manipulator;

[0009] S4, the perception phase obtains real-time information about the robot and the environment through sensors;

[0010] S5. According to the control system design, select appropriate hardware equipment to build a complete robotic arm control system;

[0011] S6. The communication system uses wireless communication technology to achieve real-time communication and data transmission between the robot and the remote control console, doctors or other equipment.

[0012] Furthermore, in the present invention, the magnetic control module includes a magnetic control principle. When the magnetically controlled miniaturized robot is in a magnetic field with a certain magnetic induction intensity, the external magnetic field and the magnetic material inside the robot generate magnetic Fm and magnetic torque Tm, wherein:

[0013]

[0014] Tm = M*B;

[0015] If the magnetization direction of the magnetic material is the same as the external magnetic field, a magnetic force is generated to drive the robot to move along the direction of the magnetic field. At this time, the magnetic material is not affected by the magnetic torque. When the magnetization direction of the magnetic material is inconsistent with the direction of the magnetic induction intensity of the external magnetic field, a magnetic torque will be generated to drive the magnetic material to deflect until its magnetization direction is collinear with the direction of the magnetic induction intensity of the external magnetic field. Magnetic force and magnetic torque are the basic principles of external magnetic field-driven magnetically controlled miniaturized robots.

[0016] Further, in the present invention, the robot arm module includes the basic principles of reinforcement learning, the basic principles of compliant control, the compliant control method, and the combination of deep reinforcement learning and compliant control;

[0017] The reinforcement learning requires a reward signal to guide the learning process, and constantly tries various actions to interact with the environment and obtain reward signals to learn and adjust its own control strategy;

[0018] The basic principle of compliance control: Applying compliance control technology on a robot arm can increase the compliance and adaptability of the robot arm;

[0019] The combination of deep reinforcement learning and compliant control: through deep reinforcement learning, the robot is trained to learn how to adaptively adjust impedance parameters so as to better adapt to different environments and task requirements when performing tasks.

[0020] Furthermore, in the present invention, the robot arm kinematics analysis and control system design, wherein the kinematics analysis comprises the following steps:

[0021] S301, establishing a kinematic model of the robot arm, in which the DH parameters of the robot arm need to be determined;

[0022] S302, calculating joint variables, where the joint variables refer to the angle or displacement of each joint of the robotic arm;

[0023] S303, calculating a homogeneous transformation matrix, and calculating a homogeneous transformation matrix of each joint according to DH parameters and joint variables, that is, a transformation matrix from a previous joint to a current joint;

[0024] The inverse kinematics analysis of the robot arm includes the following steps:

[0025] S3011, establishing a kinematic model of the robot arm. Similar to the forward kinematic analysis, in this step, the DH parameters of the robot arm need to be determined;

[0026] S3012, determining the position and posture of the end effector;

[0027] S3013, calculating joint variables;

[0028] S3014. Verify the results by simulating or actually operating the robotic arm to verify whether the calculated joint angles can achieve the desired end-effector position and posture.

[0029] Furthermore, in the present invention, the design and construction of the robotic arm control system includes control system structure design and control system platform construction. The control system structure design: uses a deep reinforcement learning method as a control system, which requires the robotic arm to continuously interact with the environment, obtain data through trial and error, and then learn control strategies; the control system platform construction includes a controller, motor, sensor, tool, communication module, power supply and signal line.

[0030] Furthermore, in the present invention, the operation flow of the electrical automation installation in the communication system and human-computer interaction operation design is as follows:

[0031] S501. On-site personnel should unload the electrical cabinet to the designated area and place it as required. The operator should issue operating instructions through the handheld terminal;

[0032] S502, the command is transmitted from the wireless communication system to the terminal control system, and the terminal control system receives the command and instructs the robot control system to execute the task;

[0033] S503, after receiving the task command, the robot control system controls the underlying drive unit to execute the task command according to the planned path;

[0034] S504, the robot starts from the standby area and moves to the workstation where the designated electrical cabinet is located according to the planned path, and automatically lifts the electrical cabinet through the tool on the top of the electrical cabinet;

[0035] S505, drive to the substation according to the planned route, and place the electrical cabinet at the installation location specified in the task description;

[0036] S506: After the robot completes the task, if there is a new command, the robot will continue to execute until no task command is issued; if there is no new command, the robot will return to the standby area and wait for the task command.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. The medical robot control system based on the deep reinforcement learning algorithm can autonomously learn how to perform tasks by interacting with the environment without the need for detailed program design in advance. Deep reinforcement learning combines the advantages of deep learning and reinforcement learning, enabling medical robots to process large amounts of high-dimensional medical data and have strong adaptability and generalization capabilities when facing complex and uncertain medical environments. The medical robot control system based on deep reinforcement learning can help the robot interact and communicate with human doctors, patients and other robots, improve the robot's intelligence and adaptability, and have a high degree of automation and intelligence.

[0039] 2. The medical robot control system is based on a deep reinforcement learning algorithm. The deep reinforcement learning algorithm can help the medical robot achieve more natural and efficient movements, especially when performing complex limbs and movements. By optimizing the medical robot's motion strategy, it can significantly improve the flexibility and accuracy of the medical robot when performing tasks such as surgery and rehabilitation training.

[0040] The present invention will be explained in detail below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a schematic diagram of the control flow of the robot control system of the present invention;

[0042] Figure 2 This is a schematic diagram of the kinematic analysis process of the robot arm of the present invention;

[0043] Figure 3 It is a schematic diagram of the inverse kinematics analysis process of the robot arm of the present invention;

[0044] Figure 4 It is a schematic diagram of the operation flow of the electrical automation installation of the present invention;

[0045] Figure 5 Schematic diagram of the robot arm of the present invention. DETAILED DESCRIPTION

[0046] To facilitate understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings, but the present invention can be implemented in different forms and is not limited to the embodiments described in the text. On the contrary, these embodiments are provided to make the content disclosed in the present invention more thorough and comprehensive.

[0047] It should be noted that when an element is referred to as being "fixed on" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used in this article are for illustrative purposes only.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which the present invention belongs. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0049] Embodiment 1:

[0050] A medical robot control system based on a deep reinforcement learning algorithm includes a magnetic control module of a magnetically controlled miniaturized medical robot, a robot mechanical arm module, a robot mechanical arm kinematic analysis and control system design, a mechanical arm control system design and construction, and a communication system and human-computer interaction operation design. The control system design based on deep reinforcement learning includes three links: perception, decision-making, and execution. The mechanical arm control system design and construction include controller design and hardware construction. The communication system and human-computer interaction operation design include communication system design and human-computer interaction operation design. The specific steps for implementing the robot control system are as follows:

[0051] S1. The magnetic control module interacts with the magnetic material inside the robot through the external magnetic field, thereby controlling the robot's motion trajectory and speed. The magnetic control module can automatically adjust the magnetic field parameters according to environmental information and task requirements to achieve adaptive control of the robot;

[0052] S2, the robot arm module is responsible for performing specific medical tasks. The robot arm module can autonomously plan the motion trajectory according to the task requirements and patient conditions to achieve precise medical operations;

[0053] S3. By analyzing the kinematic characteristics of the robot arm, the robot's kinematic model is established to provide theoretical support for the subsequent control system design;

[0054] S4, the perception phase obtains real-time information about the robot and the environment through sensors. The decision phase uses deep reinforcement learning algorithms to generate the optimal control strategy based on the perception information. The execution phase drives the robot arm to move according to the control strategy.

[0055] S5. According to the control system design, select appropriate hardware equipment, including sensors, actuators,

[0056] Motor, build a complete robotic arm control system;

[0057] S6. The communication system uses wireless communication technology to achieve real-time communication and data transmission between the robot and the remote control console, doctors or other equipment.

[0058] Embodiment 2:

[0059] According to a medical robot control system based on a deep reinforcement learning algorithm proposed in the first embodiment, the magnetic control module includes a magnetic control principle. When the magnetically controlled miniaturized robot is in a magnetic field with a certain magnetic induction intensity, the external magnetic field and the magnetic material inside the robot generate a magnetic Fm and a magnetic torque Tm, wherein:

[0060]

[0061] Tm = M*B;

[0062] If the magnetization direction of the magnetic material is the same as the external magnetic field, a magnetic force is generated to drive the robot to move in the direction of the magnetic field. At this time, the magnetic material is not affected by the magnetic torque. When the magnetization direction of the magnetic material is inconsistent with the direction of the magnetic induction intensity of the external magnetic field, a magnetic torque will be generated to drive the magnetic material to deflect until its magnetization direction is collinear with the direction of the magnetic induction intensity of the external magnetic field. Magnetic force and magnetic torque are the basic principles of external magnetic field driven magnetically controlled miniaturized robots.

[0063] Embodiment three:

[0064] According to a medical robot control system based on a deep reinforcement learning algorithm proposed in the first embodiment, the robot arm module includes the basic principles of reinforcement learning, the basic principles of compliant control, the compliant control method, and the combination of deep reinforcement learning and compliant control;

[0065] The reinforcement learning requires a reward signal to guide the learning process, and constantly tries various actions to interact with the environment and obtain reward signals to learn and adjust its own control strategy;

[0066] The basic principle of compliant control: Applying compliant control technology on a robot arm can increase the flexibility and adaptability of the robot arm, thereby improving the reliability and accuracy of the robot arm in operation. The basic problem of compliant control is the dynamic control related to force and position. According to the feedback signal, the appropriate input signal is obtained to obtain the desired movement and force.

[0067] The combination of deep reinforcement learning and compliant control: through deep reinforcement learning, the robot is trained to learn how to adaptively adjust impedance parameters so as to better adapt to different environments and task requirements when performing tasks. At the same time, impedance control can also provide better kinematic modeling, thereby helping deep reinforcement learning to better understand the interaction between the robot and the environment.

[0068] Embodiment 4:

[0069] According to a medical robot control system based on a deep reinforcement learning algorithm proposed in the first embodiment, the robot arm kinematics analysis and control system design, wherein the kinematics analysis includes the following steps:

[0070] S301, establishing a kinematic model of the robot arm. In this step, it is necessary to determine the DH parameters of the robot arm, which can be used to describe the geometric relationship between the joint axes and joints of the robot arm;

[0071] S302, calculating joint variables, where the joint variables refer to the angle or displacement of each joint of the robotic arm, which can be measured by a sensor or other device or calculated by a controller;

[0072] S303, calculating the homogeneous transformation matrix. According to the DH parameters and the joint variables, the homogeneous transformation matrix of each joint can be calculated, that is, the transformation matrix from the previous joint to the current joint. The inverse kinematics analysis of the robot arm includes the following steps:

[0073] S3011, establishing a kinematic model of the robot arm. Similar to the forward kinematic analysis, in this step, the DH parameters of the robot arm need to be determined;

[0074] S3012, determining the position and posture of the end effector, which can be obtained by manual setting or other methods;

[0075] S3013, calculating joint variables;

[0076] S3014, verification result: by simulation or actual operation of the robotic arm, it can be verified whether the calculated joint angles can achieve the desired end execution position and posture.

[0077] Embodiment five:

[0078] According to the first embodiment, a medical robot control system based on a deep reinforcement learning algorithm is proposed, wherein the design and construction of the robot arm control system includes the control system structure design and the control system platform construction. The control system structure design uses the deep reinforcement learning method as the control system, which requires the robot arm to continuously interact with the environment, obtain data through trial and error, and then learn the control strategy; the control system platform construction includes a controller, a motor, a sensor, a tool, a communication module, a power supply and a signal line.

[0079] Embodiment six:

[0080] According to a medical robot control system based on a deep reinforcement learning algorithm proposed in Example 1, the operation process of the electrical automation installation in the communication system and human-machine interactive operation design is as follows:

[0081] S501. On-site personnel should unload the electrical cabinet to the designated area and place it as required. The operator should issue operating instructions through the handheld terminal;

[0082] S502, the command is transmitted from the wireless communication system to the terminal control system, and the terminal control system receives the command and instructs the robot control system to execute the task;

[0083] S503, after receiving the task command, the robot control system controls the underlying drive unit to execute the task command according to the planned path;

[0084] S504, the robot starts from the standby area and moves to the workstation where the designated electrical cabinet is located according to the planned path, and automatically lifts the electrical cabinet through the tool on the top of the electrical cabinet;

[0085] S505, drive to the substation according to the planned route, and place the electrical cabinet at the installation location specified in the task description;

[0086] S506: After the robot completes the task, if there is a new command, the robot will continue to execute until no task command is issued; if there is no new command, the robot will return to the standby area and wait for the task command.

[0087] The above is an exemplary description of the present invention in combination with the accompanying drawings. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned method. As long as such non-substantial improvements are made by adopting the method concept and technical solution of the present invention, or the concept and technical solution of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A medical robot control system based on deep reinforcement learning algorithm, characterized by: It includes the magnetic control module of the magnetically controlled miniaturized medical robot, the robot arm module, the robot arm kinematics analysis and control system design, the robot arm control system design and construction, the communication system and human-computer interaction operation design. The control system design based on deep reinforcement learning includes three links: perception, decision-making and execution. The robot arm control system design and construction include controller design and hardware construction. The communication system and human-computer interaction operation design include communication system design and human-computer interaction operation design. The specific steps for implementing the robot control system are as follows: S1, the magnetic control module interacts with the magnetic material inside the robot through the external magnetic field to control the movement trajectory and speed of the robot; S2, the robotic arm module is responsible for performing specific medical tasks; S3. Establish a kinematic model of the robot by analyzing the kinematic characteristics of the robot manipulator; S4, the perception phase obtains real-time information about the robot and the environment through sensors; S5. According to the control system design, select appropriate hardware equipment to build a complete robotic arm control system; S6. The communication system uses wireless communication technology to achieve real-time communication and data transmission between the robot and the remote control console, doctors or other equipment.

2. A medical robot control system based on deep reinforcement learning algorithm according to claim 1, characterized in that: The magnetic control module includes a magnetic control principle. When the magnetically controlled miniaturized robot is in a magnetic field with a certain magnetic induction intensity, the external magnetic field and the magnetic material inside the robot generate magnetic Fm and magnetic torque Tm, where: Tm = M*B; If the magnetization direction of the magnetic material is the same as the external magnetic field, a magnetic force is generated to drive the robot to move along the direction of the magnetic field. At this time, the magnetic material is not affected by the magnetic torque. When the magnetization direction of the magnetic material is inconsistent with the direction of the magnetic induction intensity of the external magnetic field, a magnetic torque will be generated to drive the magnetic material to deflect until its magnetization direction is collinear with the direction of the magnetic induction intensity of the external magnetic field. Magnetic force and magnetic torque are the basic principles of external magnetic field-driven magnetically controlled miniaturized robots.

3. A medical robot control system based on deep reinforcement learning algorithm according to claim 1, characterized in that: The robot arm module includes the basic principles of reinforcement learning, the basic principles of compliant control, the compliant control method, and the combination of deep reinforcement learning and compliant control; The reinforcement learning requires a reward signal to guide the learning process, and constantly tries various actions to interact with the environment and obtain reward signals to learn and adjust its own control strategy; The basic principle of compliance control: Applying compliance control technology on a robot arm can increase the compliance and adaptability of the robot arm; The combination of deep reinforcement learning and compliant control: through deep reinforcement learning, the robot is trained to learn how to adaptively adjust impedance parameters so as to better adapt to different environments and task requirements when performing tasks.

4. A medical robot control system based on deep reinforcement learning algorithm according to claim 1, characterized in that: The robot arm kinematics analysis and control system design, wherein the kinematics analysis includes the following steps: S301, establishing a kinematic model of the robot arm, in which the DH parameters of the robot arm need to be determined; S302, calculating joint variables, where the joint variables refer to the angle or displacement of each joint of the robotic arm; S303, calculating a homogeneous transformation matrix, and calculating a homogeneous transformation matrix of each joint according to DH parameters and joint variables, that is, a transformation matrix from a previous joint to a current joint; The inverse kinematics analysis of the robot arm includes the following steps: S3011, establishing a kinematic model of the robot arm. Similar to the forward kinematic analysis, in this step, the DH parameters of the robot arm need to be determined; S3012, determining the position and posture of the end effector; S3013, calculating joint variables; S3014. Verify the results by simulating or actually operating the robotic arm to verify whether the calculated joint angles can achieve the desired end-effector position and posture.

5. A medical robot control system based on deep reinforcement learning algorithm according to claim 1, characterized in that: The design and construction of the robotic arm control system includes control system structure design and control system platform construction. The control system structure design uses a deep reinforcement learning method as a control system, which requires the robotic arm to continuously interact with the environment, obtain data through trial and error, and then learn control strategies; the control system platform construction includes a controller, motor, sensor, tool, communication module, power supply and signal line.

6. A medical robot control system based on deep reinforcement learning algorithm according to claim 1, characterized in that: The operation process of electrical automation installation in the communication system and human-machine interactive operation design is as follows: S501. On-site personnel should unload the electrical cabinet to the designated area and place it as required. The operator should issue operating instructions through the handheld terminal; S502, the command is transmitted from the wireless communication system to the terminal control system, and the terminal control system receives the command and instructs the robot control system to execute the task; S503, after receiving the task command, the robot control system controls the underlying drive unit to execute the task command according to the planned path; S504, the robot starts from the standby area and moves to the workstation where the designated electrical cabinet is located according to the planned path, and automatically lifts the electrical cabinet through the tool on the top of the electrical cabinet; S505, drive to the substation according to the planned route, and place the electrical cabinet at the installation location specified in the task description; S506: After the robot completes the task, if there is a new command, the robot will continue to execute until no task command is issued; if there is no new command, the robot will return to the standby area and wait for the task command.

Images