Control method of telemedicine robot and control system for controlling telemedicine robot

Abstract

The invention provides a control method of a telemedicine robot and a control system for controlling the telemedicine robot. The control method and the control system can be applied to the technical field of medical treatment and the technical field of mechanical control. The control method of the remote medical robot comprises the steps that in response to the fact that master-slave mapping frequency of a communication signal between a master end and a slave end does not meet a frequency threshold value, a real-time pose sequence of the slave end is extracted from control information used for controlling the slave end, and the master end is used for conducting communication signal interaction with the slave end in response to user operation; the slave end is used for controlling the telemedicine robot to execute actions based on the control information; inputting the real-time pose sequence into a target prediction model to obtain a predicted pose sequence; according to the trajectory path of the predicted pose sequence, trajectory interpolation is carried out on the predicted pose sequence, a target pose sequence is obtained, and the target master-slave mapping frequency meets a frequency threshold value; inputting the target pose sequence into a controller of the slave end to generate a control instruction; and controlling the telemedicine robot based on the control instruction.

Description

Technical Field

[0001] The present disclosure relates to the fields of medical technology and mechanical control technology, and more particularly to a control method for a remote medical robot and a control system for controlling a remote medical robot. Background Art

[0002] Remote surgery is a paradigm of minimally invasive surgery, which divides the surgical robot into a master hand operated remotely by a doctor and a slave hand executed at the surgical site. Due to communication network limitations, the communication frequency of remote surgery is much lower than the master-slave mapping frequency of the surgical robot. For example, the master-slave mapping frequency between the master hand and the slave hand can reach more than 1000 Hz. However, due to network limitations in remote surgery, the master-slave mapping frequency corresponding to the control data transmitted from the master hand to the slave hand is often less than 100 Hz, reducing the accuracy of the control data.

[0003] In the process of implementing the concept of the present disclosure, the inventors found that at least the following problems exist in the related art: the low master-slave mapping frequency in remote surgery reduces the accuracy of control data, resulting in reduced real-time performance and safety of remote surgery. Summary of the Invention

[0004] In view of the above problems, the present disclosure provides a control method for a remote medical robot and a control system for controlling a remote medical robot.

[0005] According to a first aspect of the present disclosure, there is provided a control method for a remote medical robot, including: in response to the master-slave mapping frequency of the communication signal between the master end and the slave end not meeting the frequency threshold, extracting the real-time pose sequence of the slave end from the control information for controlling the slave end, the control information being received by the slave end from the master end through the communication signal, the master end being used to interact with the slave end through the communication signal in response to a user operation, and the slave end being used to control the remote medical robot to execute actions based on the control information; inputting the real-time pose sequence into a target prediction model to obtain a predicted pose sequence at a future moment; performing trajectory interpolation on the predicted pose sequence according to the trajectory path of the predicted pose sequence to obtain a target pose sequence corresponding to the target master-slave mapping frequency, the target master-slave mapping frequency meeting the frequency threshold; inputting the target pose sequence into the controller of the slave end to generate a control instruction; and controlling the remote medical robot based on the control instruction.

[0006] According to an embodiment of the present disclosure, extracting the real-time pose sequence of the slave end from the control information for controlling the slave end includes: determining the joint angles of the slave end at multiple moments from the control information; and inputting the joint angles at the multiple moments into a kinematic model to obtain the real-time pose sequence of the slave end.

[0007] According to an embodiment of the present disclosure, performing trajectory interpolation on a predicted pose sequence according to a trajectory path of the predicted pose sequence to obtain a target pose sequence corresponding to a target master-slave mapping frequency includes: determining key points that satisfy a preset target curvature threshold from the trajectory path, where the key points characterize the trajectory direction of the trajectory path; segmenting the trajectory path according to the key points to obtain multiple trajectory sub-paths; and performing trajectory interpolation on the multiple trajectory sub-paths according to the pose change rate of the trajectory path to obtain a target pose sequence corresponding to the target master-slave mapping frequency.

[0008] According to an embodiment of the present disclosure, performing trajectory interpolation on multiple trajectory sub-paths according to the pose change rate of the trajectory path to obtain a target pose sequence corresponding to the target master-slave mapping frequency includes: performing trajectory interpolation on a trajectory sub-path according to the pose change rate of the trajectory sub-path to obtain a target trajectory sub-path; and splicing the multiple target trajectory sub-paths according to the respective time information of the multiple target trajectory sub-paths to obtain a target pose sequence corresponding to the target master-slave mapping frequency.

[0009] According to an embodiment of the present disclosure, performing trajectory interpolation on a trajectory sub-path according to the pose change rate of the trajectory sub-path to obtain a target trajectory sub-path includes: determining an interpolation path segment to be interpolated from the trajectory sub-path according to a constraint condition between the pose change rate of the trajectory sub-path and a preset pose change rate, where the constraint condition characterizes that the pose change rate is less than the preset pose change rate; determining a target interpolator algorithm according to the pose change rate of the interpolation path segment to be interpolated; and using the target interpolator algorithm to perform trajectory interpolation on the interpolation path segment to be interpolated to obtain a target trajectory sub-path.

[0010] According to an embodiment of the present disclosure, determining a target interpolator algorithm according to the pose change rate of the interpolation path segment to be interpolated includes: in response to the pose change rate of the interpolation path segment to be interpolated being linearly variable, determining that the target interpolator algorithm is a linear interpolation algorithm; and in response to the pose change rate of the interpolation path segment to be interpolated being non-linearly variable, determining that the target interpolator algorithm is a curve interpolation algorithm.

[0011] According to an embodiment of the present disclosure, the target prediction model is trained based on the following method: obtaining a historical master-end pose sequence and a historical slave-end pose sequence at a historical moment, where the historical control information generated by the master end includes the historical master-end pose sequence, and the historical control instruction generated by the slave end at the historical moment includes the historical slave-end pose sequence; inputting the historical master-end pose sequence into the prediction model to obtain a predicted pose sequence; and training the prediction model based on the error value between the predicted pose sequence and the historical slave-end pose sequence to obtain the target prediction model.

[0012] According to an embodiment of the present disclosure, training a prediction model based on an error value between a predicted pose sequence and a historical slave-end pose sequence to obtain a target prediction model includes: in response to the error value between the predicted pose sequence and the historical slave-end pose sequence being greater than an error threshold, updating model parameters of the prediction model to obtain a to-be-determined prediction model; in response to the to-be-determined prediction model processing a sample master-end pose sequence in a continuous preset period to obtain a sample slave-end pose sequence; in response to a periodic error value between the sample slave-end pose sequence and a labeled slave-end pose sequence in the continuous preset period being less than or equal to the error threshold, determining the to-be-determined prediction model as the target prediction model.

[0013] A second aspect of the present disclosure provides a control system for controlling a telemedicine robot. The control system includes: a master end, configured to generate control information for controlling a slave end in response to a control operation of a user; a slave end, configured to receive the control information sent by the master end and execute the control method of the above-mentioned telemedicine robot.

[0014] According to an embodiment of the present disclosure, the above system further includes: a mapping medical robot communicatively connected to the master end; the master end is further configured to: generate to-be-mapped control information in response to a user's control operation on the mapping medical robot; map the to-be-mapped control information to obtain control information.

[0015] A third aspect of the present disclosure provides a control device for a telemedicine robot, including: an extraction module, configured to extract a real-time pose sequence of the slave end from control information for controlling the slave end in response to a master-slave mapping frequency of a communication signal between the master end and the slave end not satisfying a frequency threshold, the control information being received by the slave end from the master end through the communication signal, the master end being configured to perform communication signal interaction with the slave end in response to a user operation, and the slave end being configured to control the telemedicine robot to perform actions based on the control information; a first input module, configured to input the real-time pose sequence into a target prediction model to obtain a predicted pose sequence at a future moment; a trajectory interpolation module, configured to perform trajectory interpolation on the predicted pose sequence according to a trajectory path of the predicted pose sequence to obtain a target pose sequence corresponding to a target master-slave mapping frequency, the target master-slave mapping frequency satisfying the frequency threshold; a second input module, configured to input the target pose sequence into a controller of the slave end to generate a control instruction; a control module, configured to control the telemedicine robot based on the control instruction.

[0016] A fourth aspect of the present disclosure provides an electronic device, including: one or more processors; a memory, configured to store one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0017] The fifth aspect of the present disclosure also provides a computer-readable storage medium, on which a computer program or instruction is stored, and when the computer program or instruction is executed by a processor, the steps of the above method are implemented.

[0018] The sixth aspect of the present disclosure also provides a computer program product, including a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps of the above method are implemented.

[0019] According to an embodiment of the present disclosure, in response to the master-slave mapping frequency of the communication signal between the master end and the slave end not meeting the frequency threshold, the real-time pose sequence of the slave end can be predicted to obtain a predicted pose sequence; trajectory interpolation is performed on the predicted pose sequence according to the trajectory path of the predicted pose sequence to obtain a target pose sequence that meets the frequency threshold. That is, it realizes the conversion of the low-frequency predicted pose sequence into a high-frequency target pose sequence, improving the accuracy of the pose sequence. The target pose sequence is input into the controller of the slave end to generate a control instruction; the remote medical robot is controlled based on the control instruction, thereby reducing the influence of delay on the movement of the remote medical robot, realizing synchronous tracking of the pose, improving the real-time performance and safety of the remote surgery, and ensuring the accuracy of the remote surgical operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Through the following description of the embodiments of the present disclosure with reference to the drawings, the above content and other objects, features and advantages of the present disclosure will become clearer. In the drawings:

[0021] Figure 1 Schematically shows an application scenario diagram of a control method for a remote medical robot according to an embodiment of the present disclosure;

[0022] Figure 2 Schematically shows a flowchart of a control method for a remote medical robot according to an embodiment of the present disclosure;

[0023] Figure 3 Schematically shows a flowchart of performing trajectory interpolation on a predicted pose sequence according to the trajectory path of the predicted pose sequence to obtain a target pose sequence corresponding to the target master-slave mapping frequency according to an embodiment of the present disclosure;

[0024] Figure 4 Schematically shows a flowchart of a training method for a target prediction model;

[0025] Figure 5 Schematically shows a structural block diagram of a control system for controlling a remote medical robot;

[0026] Figure 6 Schematically shows a structural block diagram of a control device for a remote medical robot according to an embodiment of the present disclosure; and

[0027] Figure 7 A block diagram of an electronic device suitable for implementing a control method of a telemedicine robot according to an embodiment of the present disclosure is schematically shown. Detailed implementation manners

[0028] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for the sake of explanation, many specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure. However, obviously, one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present disclosure.

[0029] The terms used herein are merely for describing specific embodiments and are not intended to limit the present disclosure. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0030] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0031] In the case of using expressions such as "at least one of A, B, and C, etc.", generally, it should be interpreted according to the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0032] In the technical solution of the present disclosure, the user information involved (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties. And the processing of relevant data, such as collection, storage, use, processing, transmission, provision, disclosure, and application, etc., all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0033] In the scenario of making automated decisions using personal information, the methods, devices, and systems provided by the embodiments of the present disclosure all provide corresponding operation entrances for users to choose to agree or reject the results of automated decisions; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision" here refers to the activity of automatically analyzing and evaluating an individual's behavior habits, hobbies, or economic, health, credit status, etc. through a computer program and making decisions. The expression "expert decision" here refers to the activity of making decisions by personnel who are engaged in work in a certain field, have specialized experience, knowledge, and skills, and have reached a certain professional level.

[0034] Embodiments of the present disclosure provide a control method for a telemedicine robot, including: in response to the master-slave mapping frequency of the communication signal between the master end and the slave end not meeting the frequency threshold, extracting the real-time pose sequence of the slave end from the control information for controlling the slave end, the control information being received by the slave end from the master end through the communication signal, the master end being used to interact with the slave end through the communication signal in response to user operations, and the slave end being used to control the telemedicine robot to perform actions based on the control information; inputting the real-time pose sequence into a target prediction model to obtain a predicted pose sequence at a future moment; performing trajectory interpolation on the predicted pose sequence according to the trajectory path of the predicted pose sequence to obtain a target pose sequence corresponding to the target master-slave mapping frequency, the target master-slave mapping frequency meeting the frequency threshold; inputting the target pose sequence into the controller of the slave end to generate a control instruction; and controlling the telemedicine robot based on the control instruction.

[0035] Figure 1 Schematically shows an application scenario diagram of the control method of the telemedicine robot according to an embodiment of the present disclosure.

[0036] As Figure 1 shown, the application scenario 100 according to this embodiment may include a master end 101, a slave end 102, a telemedicine robot 103, and a network 104. The network 104 is used to provide a medium for the communication link between the master end 101 and the slave end 102 and a medium for the communication link between the slave end 102 and the telemedicine robot 103. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0037] The master end 101 is the operation end of the user. The master end 101 has operation buttons, and the user can use the operation buttons to remotely control the telemedicine robot 103. The master end 101 is used to generate control information according to the user's operations, and the control information includes the real-time pose sequence of the slave end 102.

[0038] The slave end 102 has a server and a controller. The server can be used to extract the real-time pose sequence of the slave end 102 from the control information for controlling the slave end 102 in response to the master-slave mapping frequency of the communication signal between the master end 101 and the slave end 102 not meeting the frequency threshold. The real-time pose sequence is processed using a target prediction model to obtain a predicted pose sequence at a future moment. Trajectory interpolation is performed on the predicted pose sequence according to the trajectory path of the predicted pose sequence to obtain a target pose sequence. Then, the target pose sequence is input into the controller of the slave end to generate a control instruction. The remote medical robot is controlled based on the control instruction.

[0039] The remote medical robot 103 is used to execute the control instruction to complete the action corresponding to the control instruction. The remote medical robot can be a three-degree-of-freedom or six-degree-of-freedom robot. The remote medical robot 103 can be equipped with a variety of sensors so that the user can obtain real-time surgical feedback information. For example, the sensor can be an image acquisition device, an acceleration sensor, etc.

[0040] It should be noted that the control method of the remote medical robot provided by the embodiments of the present disclosure can generally be executed by the slave end 102. Correspondingly, the control device of the remote medical robot provided by the embodiments of the present disclosure can generally be set in the slave end 102.

[0041] It should be understood that Figure 1 the numbers of the master end 101, the slave end 102, the remote medical robot 103, and the network 104 in

[0042] It should be noted that the user operating the master end can be a professional with relevant operation qualification certifications. For example, the user can be a medical staff with medical practice qualification certifications.

[0043] In one embodiment of the present disclosure, the user can perform an operation on the master end, and the slave end can control the remote medical robot to perform other actions based on the method provided by the embodiments of the present disclosure. The embodiments of the present disclosure do not limit the specific type of the action corresponding to the control instruction.

[0044] Figure 2 Schematically shows a flowchart of the control method of the remote medical robot according to an embodiment of the present disclosure.

[0045] As Figure 2 shown, the control method of the remote medical robot in this embodiment includes operations S210 to S250.

[0046] In operation S210, in response to the master-slave mapping frequency of the communication signal between the master end and the slave end not meeting the frequency threshold, the real-time pose sequence of the slave end is extracted from the control information for controlling the slave end.

[0047] According to an embodiment of the present disclosure, in a remote-controlled surgical robot, the signal transmission frequency and data mapping relationship between a user (master end) and a remote medical robot (communicatively connected to a slave end) are described. The time synchronization and feedback mechanism between the master end control input and the slave end actuator output are described.

[0048] According to an embodiment of the present disclosure, the frequency threshold may be 100 Hz. For example, if the master-slave mapping frequency is often less than 100 Hz, it will be difficult to ensure the real-time performance of remote surgical control and the safety of surgical operations.

[0049] According to an embodiment of the present disclosure, the control information is received by the slave end from the master end through a communication signal. The master end is used to interact with the slave end through a communication signal in response to a user operation, and the slave end is used to control the remote medical robot to perform actions based on the control information.

[0050] According to an embodiment of the present disclosure, the real-time pose sequence may include information such as the sequence from the starting pose point to the ending pose point, the pose change rate, the pose acceleration, etc. The real-time pose sequence is obtained by sorting the information such as the sequence from the starting pose point to the ending pose point, the pose change rate, the pose acceleration, etc. based on timestamps.

[0051] For example, in response to a user operation, control information for generating an action "grab" is generated. During the process of the remote medical robot at the slave end executing the action "grab", a change sequence (real-time pose sequence) of each joint angle is extracted from the control information.

[0052] In operation S220, the real-time pose sequence is input into a target prediction model to obtain a predicted pose sequence at a future moment.

[0053] According to an embodiment of the present disclosure, the target prediction model may be a machine learning algorithm. For example, the target prediction model may be a neural network algorithm, a random forest algorithm, a regression algorithm, etc.

[0054] For example, the neural network algorithm can predict the real-time pose sequence by using the surgical motion change law learned through training to obtain a predicted pose sequence.

[0055] For example, the random forest algorithm can predict the real-time pose sequence through the integration of multiple decision trees.

[0056] In operation S230, according to the trajectory path of the predicted pose sequence, trajectory interpolation is performed on the predicted pose sequence to obtain a target pose sequence corresponding to the target master-slave mapping frequency.

[0057] According to an embodiment of the present disclosure, the target master-slave mapping frequency satisfies the frequency threshold.

[0058] According to an embodiment of the present disclosure, the trajectory path of the predicted pose sequence is a mathematical model that describes the continuous movement of the telemedicine robot in space. For example, the trajectory path describes the change in the position of each pose point in the predicted pose sequence over time.

[0059] For example, trajectory interpolation is performed on the predicted pose sequence without changing the direction of the trajectory path of the predicted pose sequence.

[0060] For example, according to the requirements of trajectory interpolation for real-time performance and curvature accuracy, technical indicators are clarified, including bow height accuracy, required inference time, smoothness, etc. By designing a short trajectory interpolation algorithm based on the obtained indicator requirements, data continuity, real-time performance, and interpolation accuracy can be ensured. Among them, the interpolation accuracy requirement is greater than the real-time performance requirement.

[0061] According to an embodiment of the present disclosure, the target pose sequence can be cached on the slave side. And the joint angles of the telemedicine robot in the target pose sequence are resampled according to the target master-slave mapping frequency.

[0062] In operation S240, the target pose sequence is input into the controller of the slave side to generate a control instruction.

[0063] According to an embodiment of the present disclosure, the target pose sequence is denoised using a filter, and then the target pose sequence is updated at a prediction frequency of 10 Hz to replace the past prediction values, establishing a control closed-loop to monitor the accuracy of the prediction values in real time. The filter can be a Kalman filter, a mean filter, a median filter, etc.

[0064] In operation S250, the telemedicine robot is controlled based on the control instruction.

[0065] According to an embodiment of the present disclosure, in response to the master-slave mapping frequency of the communication signal between the master side and the slave side not meeting the frequency threshold, the real-time pose sequence of the slave side can be predicted to obtain a predicted pose sequence; trajectory interpolation is performed on the predicted pose sequence according to the trajectory path of the predicted pose sequence to obtain a target pose sequence that meets the frequency threshold. That is, it realizes the conversion of the low-frequency predicted pose sequence into a high-frequency target pose sequence, improving the accuracy of the pose sequence. The target pose sequence is input into the controller of the slave side to generate a control instruction; the telemedicine robot is controlled based on the control instruction, thereby reducing the influence of delay on the movement of the telemedicine robot, realizing synchronous tracking of the pose, improving the real-time performance and safety of the remote surgery, and ensuring the accuracy of the remote surgery operation.

[0066] According to an embodiment of the present disclosure, extracting the real-time pose sequence of the slave side from the control information for controlling the slave side includes: determining the joint angles of the slave side at multiple moments from the control information; inputting the joint angles at multiple moments into the kinematic model to obtain the real-time pose sequence of the slave side.

[0067] According to an embodiment of the present disclosure, the control information may include the operation instructions of the user, the direction of the end effector of the remote medical robot, the spatial position, etc.

[0068] According to an embodiment of the present disclosure, the kinematic model may include forward kinematics and inverse kinematics. By using the kinematic model to solve the joint angles at multiple moments, the real-time joint angles of each joint of the remote medical robot connected to the slave end are obtained.

[0069] According to an embodiment of the present disclosure, the real-time pose sequence may include the real-time joint angles of each joint of the remote medical robot, the pose change rate, the pose change acceleration, the pose change frequency, etc.

[0070] According to an embodiment of the present disclosure, performing trajectory interpolation on the predicted pose sequence according to the trajectory path of the predicted pose sequence to obtain a target pose sequence corresponding to the target master-slave mapping frequency includes: determining key points that satisfy a preset target curvature threshold from the trajectory path, where the key points represent the trajectory direction of the trajectory path; segmenting the trajectory path according to the key points to obtain multiple trajectory sub-paths; and performing trajectory interpolation on the multiple trajectory sub-paths according to the pose change rate of the trajectory path to obtain a target pose sequence corresponding to the target master-slave mapping frequency.

[0071] According to an embodiment of the disclosure, the predicted pose sequence is processed using a C3 continuous corner smoothing algorithm, and the curvature of each point in the trajectory path of the predicted pose sequence is obtained.

[0072] According to an embodiment of the present disclosure, the preset target curvature threshold may be the maximum curvature in the trajectory path.

[0073] The smoothed trajectory path is segmented through the key points to obtain multiple trajectory sub-paths. And the length of each trajectory sub-path is calculated.

[0074] According to an embodiment of the present disclosure, performing trajectory interpolation on the multiple trajectory sub-paths according to the pose change rate of the trajectory path to obtain a target pose sequence corresponding to the target master-slave mapping frequency includes: performing trajectory interpolation on the trajectory sub-path according to the pose change rate of the trajectory sub-path to obtain a target trajectory sub-path; and splicing the multiple target trajectory sub-paths according to the respective time information of the multiple target trajectory sub-paths to obtain a target pose sequence corresponding to the target master-slave mapping frequency.

[0075] According to embodiments of the present disclosure, multiple trajectory sub-paths in different trajectory directions are respectively subjected to trajectory interpolation. Without affecting the overall path direction, accurate data filling can be achieved. Then, multiple target trajectory sub-paths are spliced according to time information to obtain a high-quality target pose sequence. The target pose sequence can be cached at the slave end, and the cached data is resampled, and the low-frequency data sent is converted into high-frequency data and sent to the signal receiving end to achieve precise remote surgery control with fast response.

[0076] According to embodiments of the present disclosure, performing trajectory interpolation on a trajectory sub-path according to the pose change rate of the trajectory sub-path to obtain a target trajectory sub-path includes: determining an interpolation path segment to be interpolated from the trajectory sub-path according to the constraint condition between the pose change rate of the trajectory sub-path and a preset pose change rate, where the constraint condition indicates that the pose change rate is less than the preset pose change rate; determining a target interpolator algorithm according to the pose change rate of the interpolation path segment to be interpolated; and using the target interpolator algorithm to perform trajectory interpolation on the interpolation path segment to be interpolated to obtain a target trajectory sub-path.

[0077] According to embodiments of the present disclosure, the preset pose change rate may be a pose change rate limit value determined according to limitations such as chord error, feed rate command, pose acceleration, and jerk.

[0078] Based on the constraint condition of the preset pose change rate, first optimize the trajectory sub-path using a cubic acceleration curve; divide the trajectory sub-path into multiple interpolation path segments to be interpolated.

[0079] Meanwhile, the displacement amount within the interpolation path segment to be interpolated can be determined by the pose change rate of each point in the trajectory sub-path.

[0080] According to embodiments of the present disclosure, determining a target interpolator algorithm according to the pose change rate of the interpolation path segment to be interpolated includes: in response to the pose change rate of the interpolation path segment to be interpolated being linearly changed, determining the target interpolator algorithm as a linear interpolation algorithm; in response to the pose change rate of the interpolation path segment to be interpolated being non-linearly changed, determining the target interpolator algorithm as a curve interpolation algorithm.

[0081] According to embodiments of the present disclosure, the curve interpolation algorithm may be a PH spline curve interpolation algorithm.

[0082] By judging the change rule of the pose change rate of the interpolation path segment to be interpolated, select a linear interpolation algorithm or a PH spline curve interpolation algorithm to obtain a target trajectory sub-path with more accurate curvature change. Then, according to the respective time information of multiple target trajectory sub-paths, splice the multiple target trajectory sub-paths to obtain a target pose sequence with smooth trajectory and accurate data.

[0083] Finally, output the interpolated target pose sequence at a frequency that conforms to the master-slave mapping of remote surgery, reproduce the motion law of the master end, enhance the master-slave mapping, and ensure the real-time requirement of the control system.

[0084] The prediction step and the trajectory interpolation step should ensure a certain real-time performance, such as completing the prediction and trajectory interpolation inference in a sufficiently short time, and immediately outputting the inference results to the controller of the slave end at a specific frequency to ensure the stability, transparency, and smoother real-time synchronization tracking effect of the control system.

[0085] Figure 3 Schematically shows a flowchart of obtaining a target pose sequence corresponding to a target master-slave mapping frequency by performing trajectory interpolation on a predicted pose sequence according to a trajectory path of the predicted pose sequence according to an embodiment of the present disclosure.

[0086] As Figure 3 shown, the obtaining of the target pose sequence corresponding to the target master-slave mapping frequency by performing trajectory interpolation on the predicted pose sequence according to the trajectory path of the predicted pose sequence in this embodiment includes operations S301 to S309.

[0087] In operation S301, key points that meet a preset target curvature threshold are determined from the trajectory path.

[0088] In operation S302, the trajectory path is segmented according to the key points to obtain multiple trajectory sub-paths.

[0089] In operation S303, based on the constraint conditions of a preset pose change rate, the trajectory sub-paths are optimized using a cubic acceleration curve to obtain optimized trajectory sub-paths.

[0090] In operation S304, the optimized trajectory sub-paths are divided into multiple path segments to be interpolated.

[0091] In operation S305, it is determined whether the pose change rate of the path segment to be interpolated is linearly changing. If so, operation S306 is executed; if not, operation S307 is executed.

[0092] In operation S306, the linear interpolation algorithm is used as the target interpolator algorithm.

[0093] In operation S307, the curve interpolation algorithm is used as the target interpolator algorithm.

[0094] In operation S308, the target interpolator algorithm is used to perform trajectory interpolation on the path segment to be interpolated to obtain target trajectory sub-paths.

[0095] In operation S309, according to the time information of each of the multiple target trajectory sub-paths, the multiple target trajectory sub-paths are spliced to obtain a target pose sequence.

[0096] According to an embodiment of the present disclosure, the target prediction model is trained based on the following method: obtaining a historical master end pose sequence and a historical slave end pose sequence at a historical moment, where the historical control information generated by the master end includes the historical master end pose sequence, and the historical control instruction generated by the slave end at the historical moment includes the historical slave end pose sequence; inputting the historical master end pose sequence into the prediction model to obtain a predicted pose sequence; training the prediction model based on the error value between the predicted pose sequence and the historical slave end pose sequence to obtain the target prediction model.

[0097] According to an embodiment of the present disclosure, both the historical master end pose sequence and the historical slave end pose sequence are obtained by cleaning and preprocessing the original data set. For example, removing the data with duplicate records in the original data set to ensure data uniqueness by checking and deleting; using the Newton interpolation method to fill in the missing items in the original data set. Checking and processing outliers through statistical methods (z_score) and removing the noise data in the original data set through wavelet filtering (the threshold of wavelet filtering is determined by the soft threshold function), so as to obtain high-quality historical master end pose sequences and historical slave end pose sequences.

[0098] According to an embodiment of the present disclosure, the historical master end pose sequence includes the pose in the Cartesian coordinate system, the pose change frequency, the pose change rate, the pose change acceleration, the pose information of other joints in the remote medical robot, etc.

[0099] According to an embodiment of the present disclosure, the prediction model can be a neural network model. For example, the neural network can be based on time series for prediction, and the neural network framework can be a multi-layer perceptron model TimeMixer.

[0100] According to an embodiment of the present disclosure, the historical master end pose sequence and the historical slave end pose sequence are decomposed at multiple scales in the time series, that is, the information of the historical master end pose sequence and the historical slave end pose sequence is refined. For example, using DFT (Discrete Fourier Transform) to decompose the historical master end pose sequence and the historical slave end pose sequence of the multi-period time series to obtain the seasonal and trend change rules. Predict by aggregating the information of the multi-scale sequence combined with the pose characteristics.

[0101] To prevent model overfitting and improve the model generalization ability, the exponential decay algorithm is used to update the learning rate:

[0102]

[0103] where, is the initial learning rate, $\eta_t$ is the learning rate at training step $t$, where $t$ is the current training step and $\lambda$ is the decay rate that controls the speed of the learning rate decrease. A certain dropout is set to ensure the generalization ability of the model, and the loss function is set to soft-DTW (Dynamic Time Warping), which provides an output closer to the true value than MSE (Mean Squared Error). A certain amount of the dataset is used as the test set to evaluate the training results, and the values of the model output are evaluated according to evaluation metrics such as DTW (Dynamic Time Warping), MSE, and RMSE (Root Mean Square Error) to optimize the model and fine-tune the parameters.

[0104] According to an embodiment of the present disclosure, the error value can be the average error between the predicted pose sequence (predicted value) and the historical slave-end pose sequence (actual value). After the error value is less than a certain value, the target prediction model is deployed to the slave-end host computer for real-time prediction.

[0105] According to an embodiment of the present disclosure, training a prediction model based on the error value between the predicted pose sequence and the historical slave-end pose sequence to obtain a target prediction model includes: in response to the error value between the predicted pose sequence and the historical slave-end pose sequence being greater than the error threshold, updating the model parameters of the prediction model to obtain a to-be-determined prediction model; in response to the to-be-determined prediction model processing the sample master-end pose sequence in a continuous preset period to obtain a sample slave-end pose sequence; in response to the periodic error value between the sample slave-end pose sequence and the labeled slave-end pose sequence in the continuous preset period being less than or equal to the error threshold, determining the to-be-determined prediction model as the target prediction model.

[0106] If the error value between the predicted pose sequence and the historical slave-end pose sequence is within the tolerance range, continuously output and update the predicted value. If the error value between the predicted value and the true value is greater than the tolerance range (greater than the error threshold), start incremental training to update the weights of the neural network model of the surgical intention modeling module in real time, enabling the neural network model to flexibly adapt to the current status of the surgical environment. If the periodic errors between the sample slave-end pose sequence and the labeled slave-end pose sequence in the continuous preset period are all within the tolerance range (less than or equal to the error threshold), stop the incremental training and save the prediction model. If the periodic errors between the sample slave-end pose sequence and the labeled slave-end pose sequence in the continuous preset period exceed the tolerance range, continue the incremental training and update the weights.

[0107] According to an embodiment of the present disclosure, the control method of the telemedical robot is suitable for open-loop remote surgery control and also applicable to remote surgery control with force feedback. This control method is a brand-new predictive control paradigm for a telecontrol system and a novel frequency amplification idea. It intelligently intervenes in the pose signals received from the operating end at future moments, reduces the influence of delay on movement, realizes synchronous tracking of poses, and improves the mapping frequency of remote surgery while ensuring the real-time performance and accuracy of remote surgery control.

[0108] Figure 4 The flowchart of the training method of the target prediction model is schematically shown.

[0109] As Figure 4 shown, the training method of the target prediction model of this embodiment includes operations S401 to S407.

[0110] In operation S401, obtain the historical master-end pose sequence and the historical slave-end pose sequence at the historical moment. The historical control information generated by the master end includes the historical master-end pose sequence, and the historical control instruction generated by the slave end at the historical moment includes the historical slave-end pose sequence.

[0111] In operation S402, input the historical master-end pose sequence into the prediction model to obtain a predicted pose sequence.

[0112] According to an embodiment of the present disclosure, input the historical master-end pose sequence into the prediction model to obtain an interpolation-to-be pose sequence. Perform trajectory interpolation on the interpolation-to-be pose sequence to obtain a predicted pose sequence with a target master-slave mapping frequency, and the target master-slave mapping frequency meets the frequency threshold.

[0113] The predicted pose sequence and the historical slave-end pose sequence have the same master-slave mapping frequency, thereby reducing the data error caused by lag distortion.

[0114] In operation S403, determine whether the error value between the predicted pose sequence and the historical slave-end pose sequence is greater than the error threshold. If so, execute operation S404; if not, execute operation S407. In operation S404, update the model parameters of the prediction model to obtain a to-be-determined prediction model.

[0115] In operation S405, in response to the to-be-determined prediction model processing the sample master-end pose sequence in a continuous preset period, obtain a sample slave-end pose sequence.

[0116] In operation S406, determine whether the periodic error value between the sample slave-end pose sequence and the labeled slave-end pose sequence in the continuous preset period is less than or equal to the error threshold. If not, execute operation S401; if so, execute operation S407.

[0117] In operation S407, determine the to-be-determined prediction model as the target prediction model.

[0118] Figure 5 Schematically shows a structural block diagram of a control system for controlling a telemedicine robot.

[0119] As Figure 5 shown, the control system 500 for controlling a telemedicine robot according to this embodiment includes a master end 101 and a slave end 102.

[0120] The master end 101 is configured to generate control information for controlling the slave end 102 in response to a user's control operation.

[0121] The slave end 102 is configured to receive the control information sent by the master end 101 and execute the above-mentioned control method of the telemedicine robot.

[0122] According to an embodiment of the present disclosure, the master end 101 may be an operation end of the user. The master end 101 has operation buttons, and the user can remotely control the telemedicine robot 103 by using the operation buttons.

[0123] According to an embodiment of the present disclosure, the slave end 102 may be communicatively connected to the master end 101. Receive control information from the master end 101.

[0124] The slave end 102 is configured to extract the real-time pose sequence of the slave end 102 from the control information for controlling the slave end 102 in response to the master-slave mapping frequency of the communication signal between the master end 101 and the slave end 102 not meeting the frequency threshold. Process the real-time pose sequence by using a target prediction model to obtain a predicted pose sequence at a future moment. Perform trajectory interpolation on the predicted pose sequence according to the trajectory path of the predicted pose sequence to obtain a target pose sequence. Then input the target pose sequence into the controller of the slave end 102 to generate a control instruction. Control the telemedicine robot based on the control instruction.

[0125] The slave end 102 may include a server and a controller.

[0126] The server may be configured to execute the above-mentioned control method of the telemedicine robot.

[0127] The controller may be configured to generate a control instruction according to the target pose sequence.

[0128] According to an embodiment of the present disclosure, the above system further includes: a mapping medical robot, communicatively connected to the master end; the master end is further configured to: generate to-be-mapped control information in response to a user's control operation on the mapping medical robot; map the to-be-mapped control information to obtain control information.

[0129] According to an embodiment of the present disclosure, the mapping medical robot and the telemedical robot have the same degrees of freedom. The user indirectly controls the telemedical robot by controlling the mapping medical robot to maintain the intuitive and precise operation of the doctor.

[0130] In order to reproduce the operation of the user on the mapping medical robot, in response to the user's control operation on the mapping medical robot, to-be-mapped control information is generated; the to-be-mapped control information is mapped to obtain control information for controlling the telemedical robot.

[0131] Figure 6 The structural block diagram of the control device of the telemedical robot according to an embodiment of the present disclosure is schematically shown.

[0132] As Figure 6 shown, the control device 600 of the telemedical robot in this embodiment includes an extraction module 610, a first input module 620, a trajectory interpolation module 630, a second input module 640, and a control module 650.

[0133] The extraction module 610 is configured to extract a real-time pose sequence of the slave end from the control information for controlling the slave end in response to the master-slave mapping frequency between the master end and the slave end not satisfying the frequency threshold. The control information is received by the slave end from the master end through a communication signal. The master end is configured to perform communication signal interaction with the slave end in response to a user operation. The slave end is configured to control the telemedical robot to perform actions based on the control information. In an embodiment, the extraction module 610 may be configured to perform the operation S210 described above, which will not be elaborated herein.

[0134] The first input module 620 is configured to input the real-time pose sequence into a target prediction model to obtain a predicted pose sequence at a future moment. In an embodiment, the first input module 620 may be configured to perform the operation S220 described above, which will not be elaborated herein.

[0135] The trajectory interpolation module 630 is configured to perform trajectory interpolation on the predicted pose sequence according to the trajectory path of the predicted pose sequence to obtain a target pose sequence corresponding to the target master-slave mapping frequency, where the target master-slave mapping frequency satisfies the frequency threshold. In an embodiment, the trajectory interpolation module 630 may be configured to perform the operation S230 described above, which will not be elaborated herein.

[0136] The second input module 640 is configured to input the target pose sequence into the controller of the slave end to generate a control instruction. In an embodiment, the second input module 640 may be configured to perform the operation S240 described above, which will not be elaborated herein.

[0137] The control module 650 is configured to control the remote medical robot based on a control instruction. In one embodiment, the control module 650 may be configured to perform the operation S250 described above, which will not be elaborated herein.

[0138] According to an embodiment of the present disclosure, the extraction module 610 includes a first determination sub-module and a first input sub-module. The first determination sub-module is configured to determine the joint angles of the slave end at multiple moments from the control information; the first input sub-module is configured to input the joint angles at multiple moments into the kinematic model to obtain the real-time pose sequence of the slave end.

[0139] According to an embodiment of the present disclosure, the trajectory interpolation module 630 includes a second determination sub-module, a segmentation sub-module, and a trajectory interpolation sub-module. The second determination sub-module is configured to determine key points that meet a preset target curvature threshold from the trajectory path, and the key points characterize the trajectory direction of the trajectory path; the segmentation sub-module is configured to segment the trajectory path according to the key points to obtain multiple trajectory sub-paths; the trajectory interpolation sub-module is configured to perform trajectory interpolation on the multiple trajectory sub-paths according to the pose change rate of the trajectory path to obtain a target pose sequence corresponding to the target master-slave mapping frequency.

[0140] According to an embodiment of the present disclosure, the trajectory interpolation sub-module includes a trajectory interpolation unit and a splicing unit. The trajectory interpolation unit is configured to perform trajectory interpolation on the trajectory sub-path according to the pose change rate of the trajectory sub-path to obtain a target trajectory sub-path; the splicing unit is configured to splice the multiple target trajectory sub-paths according to the respective time information of the multiple target trajectory sub-paths to obtain a target pose sequence corresponding to the target master-slave mapping frequency.

[0141] According to an embodiment of the present disclosure, the trajectory interpolation unit further includes a first determination sub-unit, a second determination sub-unit, and a trajectory interpolation sub-unit. The first determination sub-unit is configured to determine an interpolation path segment to be interpolated from the trajectory sub-path according to a constraint condition between the pose change rate of the trajectory sub-path and a preset pose change rate, and the constraint condition characterizes that the pose change rate is less than the preset pose change rate; the second determination sub-unit is configured to determine a target interpolator algorithm according to the pose change rate of the interpolation path segment to be interpolated; the trajectory interpolation sub-unit is configured to perform trajectory interpolation on the interpolation path segment to be interpolated by using the target interpolator algorithm to obtain a target trajectory sub-path.

[0142] According to an embodiment of the present disclosure, the second determination sub-unit includes: in response to the pose change rate of the interpolation path segment to be interpolated being linearly changed, determining that the target interpolator algorithm is a linear interpolation algorithm; in response to the pose change rate of the interpolation path segment to be interpolated being non-linearly changed, determining that the target interpolator algorithm is a curve interpolation algorithm.

[0143] According to an embodiment of the present disclosure, the target prediction model is trained based on the following: obtaining a historical master end pose sequence and a historical slave end pose sequence at a historical moment, where the historical control information generated by the master end includes the historical master end pose sequence, and the historical control instruction generated by the slave end at the historical moment includes the historical slave end pose sequence; inputting the historical master end pose sequence into a prediction model to obtain a predicted pose sequence; and training the prediction model based on the error value between the predicted pose sequence and the historical slave end pose sequence to obtain the target prediction model.

[0144] According to an embodiment of the present disclosure, training the prediction model based on the error value between the predicted pose sequence and the historical slave end pose sequence to obtain the target prediction model includes: in response to the error value between the predicted pose sequence and the historical slave end pose sequence being greater than an error threshold, updating the model parameters of the prediction model to obtain a to-be-determined prediction model; in response to the to-be-determined prediction model processing a sample master end pose sequence in a continuous preset period to obtain a sample slave end pose sequence; and in response to the periodic error value between the sample slave end pose sequence and the labeled slave end pose sequence in the continuous preset period being less than or equal to the error threshold, determining the to-be-determined prediction model as the target prediction model.

[0145] According to an embodiment of the present disclosure, any multiple of the extraction module 610, the first input module 620, the trajectory interpolation module 630, the second input module 640, and the control module 650 may be combined and implemented in one module, or any one of them may be split into multiple modules. Or, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the extraction module 610, the first input module 620, the trajectory interpolation module 630, the second input module 640, and the control module 650 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or may be implemented by any other reasonable means such as integrating or packaging the circuit, etc., in hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Or, at least one of the extraction module 610, the first input module 620, the trajectory interpolation module 630, the second input module 640, and the control module 650 may be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding functions may be executed.

[0146] Figure 7 A block diagram of an electronic device suitable for implementing the control method of a remote medical robot according to an embodiment of the present disclosure is schematically shown.

[0147] AsFigure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage section 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (e.g., an application specific integrated circuit (ASIC)), etc. The processor 701 may also include on-board memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0148] In the RAM 703, various programs and data required for the operation of the electronic device 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. The processor 701 performs various operations of the method flow according to an embodiment of the present disclosure by executing the programs in the ROM 702 and / or the RAM 703. It should be noted that the program may also be stored in one or more memories other than the ROM 702 and the RAM 703. The processor 701 may also perform various operations of the method flow according to an embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0149] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, and the input / output (I / O) interface 705 is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read from it can be installed into the storage section 708 as needed.

[0150] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist alone without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the methods according to the embodiments of the present disclosure are implemented.

[0151] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the above-described ROM 702 and / or RAM 703 and / or one or more memories other than ROM 702 and RAM 703.

[0152] Embodiments of the present disclosure also include a computer program product, which includes a computer program, and the computer program contains program codes for executing the methods shown in the flowcharts. When the computer program product runs in a computer system, the program codes are used to enable the computer system to implement the control method of the remote medical robot provided by the embodiments of the present disclosure.

[0153] When the computer program is executed by the processor 701, the above functions defined in the system / apparatus of the embodiments of the present disclosure are executed. According to an embodiment of the present disclosure, the above-described systems, apparatuses, modules, units, etc. may be implemented by computer program modules.

[0154] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and be downloaded and installed through the communication part 709, and / or be installed from the removable medium 711. The program codes included in the computer program may be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0155] In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above functions defined in the system of the embodiments of the present disclosure are executed. According to the embodiments of the present disclosure, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0156] According to the embodiments of the present disclosure, the program code for executing the computer program provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, such as Java, C++, python, the "C" language, or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0157] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0158] Those skilled in the art can understand that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0159] The embodiments of the present disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments have been described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and all of these substitutions and modifications should fall within the scope of the present disclosure.

Claims

1. A control method for a telemedicine robot, characterized in that: The method comprises: In response to the master-slave mapping frequency of the communication signal between the master and the slave not satisfying the frequency threshold, extracting the real-time posture sequence of the slave from the control information used to control the slave, wherein the control information is received by the slave from the master through the communication signal, the master is used to interact with the slave by communication signals in response to user operations, and the slave is used to control the remote medical robot to perform actions based on the control information; Inputting the real-time pose sequence into a target prediction model to obtain a predicted pose sequence at a future time; According to the trajectory path of the predicted posture sequence, trajectory interpolation is performed on the predicted posture sequence to obtain a target posture sequence corresponding to a target master-slave mapping frequency, wherein the target master-slave mapping frequency satisfies the frequency threshold; Inputting the target posture sequence into the controller of the slave end to generate a control instruction; and The telemedicine robot is controlled based on the control instruction.

2. The method according to claim 1, characterized in that The step of extracting the real-time posture sequence of the slave terminal from the control information used to control the slave terminal comprises: Determining the joint angles of the slave end at multiple moments from the control information; The joint angles at the multiple moments are input into a kinematic model to obtain the real-time posture sequence of the slave end.

3. The method according to claim 1, characterized in that The step of performing trajectory interpolation on the predicted pose sequence according to the trajectory path of the predicted pose sequence to obtain a target pose sequence corresponding to the target master-slave mapping frequency comprises: Determining a key point that satisfies a preset target curvature threshold from the trajectory path, wherein the key point represents a trajectory direction of the trajectory path; Segmenting the trajectory path according to the key points to obtain a plurality of trajectory sub-paths; Trajectory interpolation is performed on the plurality of trajectory sub-paths according to the posture change rate of the trajectory path to obtain the target posture sequence corresponding to the target master-slave mapping frequency.

4. The method according to claim 3, characterized in that The step of performing trajectory interpolation on the plurality of trajectory sub-paths according to the posture change rate of the trajectory path to obtain the target posture sequence corresponding to the target master-slave mapping frequency includes: Performing trajectory interpolation on the trajectory sub-path according to the posture change rate of the trajectory sub-path to obtain a target trajectory sub-path; According to the respective time information of the multiple target trajectory sub-paths, the multiple target trajectory sub-paths are spliced ​​to obtain the target posture sequence corresponding to the target master-slave mapping frequency.

5. The method according to claim 4, characterized in that The step of performing trajectory interpolation on the trajectory sub-path according to the posture change rate of the trajectory sub-path to obtain a target trajectory sub-path comprises: Determine a path segment to be interpolated from the trajectory sub-path according to a constraint condition between a posture change rate of the trajectory sub-path and a preset posture change rate, wherein the constraint condition indicates that the posture change rate is less than the preset posture change rate; Determining a target interpolator algorithm according to the pose change rate of the path segment to be interpolated; The target interpolator algorithm is used to perform trajectory interpolation on the path segment to be interpolated to obtain the target trajectory sub-path.

6. The method according to claim 5, characterized in that Determining a target interpolator algorithm according to the pose change rate of the path segment to be interpolated comprises: In response to the pose change rate of the to-be-interpolated path segment being a linear change, determining that the target interpolator algorithm is a linear interpolation algorithm; In response to the posture change rate of the path segment to be interpolated being a nonlinear change, the target interpolator algorithm is determined to be a curve interpolation algorithm.

7. The method according to claim 1, characterized in that The target prediction model is trained based on the following method: Acquire a historical master-end pose sequence and a historical slave-end pose sequence at a historical moment, wherein the historical control information generated by the master end includes the historical master-end pose sequence, and the historical control instruction generated by the slave end at a historical moment includes the historical slave-end pose sequence; Inputting the historical master-end pose sequence into a prediction model to obtain a predicted pose sequence; The prediction model is trained based on the error value between the predicted posture sequence and the historical slave end posture sequence to obtain a target prediction model.

8. The method according to claim 7, characterized in that The step of training the prediction model based on the error value between the predicted pose sequence and the historical slave pose sequence to obtain the target prediction model comprises: In response to an error value between the predicted posture sequence and the historical slave end posture sequence being greater than an error threshold, updating a model parameter of the prediction model to obtain a prediction model to be determined; In response to the prediction model to be determined, processing the sample master end pose sequence in a continuous preset period, obtaining the sample slave end pose sequence; In response to the periodic error value between the sample slave end pose sequence and the tag slave end pose sequence in the continuous preset period being less than or equal to the error threshold, the prediction model to be determined is determined as the target prediction model.

9. A control system for controlling a telemedicine robot, characterized in that: The control system comprises: The master end is used to generate control information for controlling the slave end in response to the control operation of the user; The slave end is used to receive the control information sent by the master end and execute the method described in any one of claims 1 to 8.

10. The system according to claim 9, characterized in that The system further comprises: Mapping the medical robot to communicate with the master end; The master end is further used for: generating control information to be mapped in response to the user performing a control operation on the mapping medical robot; and mapping the control information to be mapped to obtain the control information.

Images