Method and apparatus for handling motion latency for remote surgery

Abstract

The present disclosure provides a processing method and device for time delay of remote operation movement, equipment and storage medium, which can be applied to the field of artificial intelligence technology. The method comprises the following steps: acquiring joint pose data of a master hand end of a remote surgery robot according to a target frequency; for each acquired joint pose data, predicting delayed joint pose data according to the joint pose data; evaluating the influence of time delay on remote operation movement according to the joint pose data and joint pose data before time delay, and obtaining an evaluation result; and performing weighted compensation on the delayed joint pose data according to the evaluation result, so as to realize compensation for remote operation movement.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, medium, and program product for processing motion delays during remote surgical operations. Background Technology

[0002] The development of remote surgery is rapid, and latency is a critical issue. Latency affects the accuracy and smoothness of surgery, increasing surgical risks. Because of issues such as sudden jitter, packet loss, and uncertainty in signal transmission during remote surgery, the risk of sudden surgical accidents increases significantly. Therefore, how to assess and compensate for motion latency in remote surgical operations is a technical problem that needs to be solved in related technologies. Summary of the Invention

[0003] In view of the above problems, this disclosure provides a method, apparatus, device, medium and program product for processing motion delays in remote surgical operations.

[0004] According to a first aspect of this disclosure, a method for processing motion delays in remote surgical operations is provided, comprising:

[0005] Acquire joint pose data of the master end of the remote surgical robot at the target frequency;

[0006] For each acquired joint pose data, the delayed joint pose data is predicted based on the acquired joint pose data.

[0007] Based on the joint pose data mentioned above and the joint pose data before the delay size, the impact of the delay on the movement of remote surgical operations is evaluated, and the evaluation results are obtained.

[0008] Based on the above evaluation results, the delayed joint pose data are weighted and compensated to compensate for the movements of the above remote surgical operation.

[0009] According to embodiments of this disclosure, the above-mentioned prediction of delayed joint pose data based on the acquired joint pose data for each time includes:

[0010] Using the delay model equation, the delay magnitude of the above joint pose data is predicted, and the predicted delay magnitude is obtained; and

[0011] Using a neural network, based on the predicted delay and the joint pose data, the delayed joint pose data is predicted.

[0012] According to embodiments of this disclosure, the above-mentioned evaluation of the impact of time delay on remote surgical operation movements based on the joint pose data and the joint pose data before the time delay is determined, and the evaluation results are obtained, including:

[0013] Based on the joint pose data mentioned above and the joint pose data before the delay magnitude, motion evaluation indicators were calculated; and

[0014] Based on the above motion assessment indicators, the impact of the delay on the motion of the above remote surgical operation was assessed, and the above assessment results were obtained.

[0015] According to embodiments of this disclosure, the motion evaluation index includes a motion evaluation index before the delay and a motion evaluation index after the delay. The motion evaluation index is calculated based on the joint pose data and the joint pose data before the delay, including:

[0016] The joint pose data acquired each time and the joint pose data before the aforementioned delay size are smoothed to obtain the first joint pose data and the second pose data; and

[0017] The first joint pose data and the second joint pose data are processed to obtain the motion evaluation index before the delay corresponding to the first joint pose data and the motion evaluation index after the delay corresponding to the second joint pose data.

[0018] According to embodiments of this disclosure, the above-mentioned evaluation results include a first evaluation result and a second evaluation result. The above-mentioned evaluation results, obtained by assessing the impact of delay on the remote surgical procedure movement based on the above-mentioned motion evaluation indicators, include:

[0019] Based on the motion evaluation indicators before and after the delay, the difference between each indicator before and after the delay is calculated to obtain the first evaluation result.

[0020] For each metric, obtain the time series before and after the delay, where the time series includes the target quantity of the metric; and

[0021] The similarity between the time series before and after the delay is calculated to obtain the second evaluation result.

[0022] According to embodiments of this disclosure, the weighted compensation of the delayed joint pose data based on the evaluation results to compensate for the motion of the remote surgical operation includes:

[0023] Based on the comprehensive evaluation results, the weights corresponding to the aforementioned delayed joint pose data are determined, wherein the aforementioned comprehensive evaluation results include the aforementioned evaluation results; and

[0024] Based on the delayed joint pose data and the weights mentioned above, the target joint pose data is obtained.

[0025] According to embodiments of this disclosure, the above-described method for handling motion delays in remote surgical operations further includes:

[0026] For each acquired joint pose data, the delay corresponding to the joint pose data is calculated based on the signal transmission time and signal reception time corresponding to the joint pose data.

[0027] Based on the delay standard deviation determined by the target number of joint pose data and the delay magnitude corresponding to the above joint pose data, the delay jitter magnitude corresponding to the above joint pose data is calculated.

[0028] The joint pose data mentioned above are matched to obtain the matching results; and

[0029] Based on the matching results above, the number of packet losses corresponding to the joint pose data above is determined.

[0030] A second aspect of this disclosure provides a device for processing motion delays during remote surgical operations, comprising: an acquisition module, a prediction module, an evaluation module, and a compensation module. The acquisition module is used to acquire joint pose data of the master end of a remote surgical robot at a target frequency. The prediction module is used to predict delayed joint pose data based on the acquired joint pose data for each iteration. The evaluation module is used to evaluate the impact of the delay on the motion of the remote surgical operation based on the acquired joint pose data and the joint pose data before the delay, obtaining an evaluation result. The compensation module is used to perform weighted compensation on the delayed joint pose data based on the evaluation result, thereby compensating for the motion delay during the remote surgical operation.

[0031] A third aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the methods described above.

[0032] A fourth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the methods described above.

[0033] The fifth aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0034] According to the method, apparatus, device, medium, and program product for processing motion delay in remote surgical operations provided in this disclosure, for each acquired joint pose data of the master end of the remote surgical robot, the delayed joint pose data is predicted based on the joint pose data. The impact of the delay on the remote surgical operation motion can be evaluated based on the joint pose data and the joint pose data before the delay, and an evaluation result can be obtained. This allows for real-time evaluation of the impact of network latency on the remote surgical operation motion, identification of potential risks, assessment of master-slave consistency between the master and slave ends, and weighted compensation of the delayed joint pose data based on the evaluation result to compensate for the remote surgical operation motion, thereby reducing the impact of network latency and improving surgical outcomes. Attached Figure Description

[0035] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0036] Figure 1 This diagram illustrates an application scenario of a method for handling motion delays in remote surgical operations according to an embodiment of the present disclosure.

[0037] Figure 2 A flowchart illustrating a method for handling motion delays in remote surgical operations according to an embodiment of the present disclosure is shown schematically.

[0038] Figure 3 The illustration shows a schematic diagram of an evaluation prediction compensation procedure in a remote surgical robot system according to an embodiment of the present disclosure;

[0039] Figure 4 The illustration shows a screenshot of the dynamic window operation of the display interface of the host computer according to an embodiment of the present disclosure;

[0040] Figure 5 A schematic diagram illustrating the time sequence before and after a delay according to an embodiment of the present disclosure is shown.

[0041] Figure 6 A schematic diagram illustrates a structural block diagram of a device for processing motion delays in remote surgical operations according to an embodiment of the present disclosure; and

[0042] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a method for processing motion delays in remote surgical operations according to an embodiment of the present disclosure. Detailed Implementation

[0043] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0044] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0045] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0046] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0047] In the technical solution of this invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0048] In implementing this disclosure, it was discovered that delays in remote surgery can be divided into master-slave sampling execution delay and display capture delay, where the master-slave sampling execution delay can include network transmission delay of control signals. Currently, it is difficult to establish a new control system with significant performance advantages; therefore, some researchers have turned to understanding the basic operation of digital communication network systems. Due to the jitter and uncertainty of network delays, the risk of sudden surgical accidents increases significantly. Furthermore, there is no consensus on the safe range of signal delays; simply limiting the safe range of remote surgery delays to a fixed range is insufficient to meet the needs of specific situations. Therefore, how to assess and compensate for motion delays in remote surgical operations is a technical problem that needs to be solved in related technologies.

[0049] Therefore, embodiments of this disclosure provide a method for processing motion delay during remote surgical operations, including: acquiring joint pose data of the master end of a remote surgical robot at a target frequency; predicting delayed joint pose data based on the acquired joint pose data; evaluating the impact of the delay on the motion of the remote surgical operation based on the joint pose data and the joint pose data before the delay, and obtaining an evaluation result; and performing weighted compensation on the delayed joint pose data based on the evaluation result to achieve compensation for the motion of the remote surgical operation.

[0050] Figure 1 The illustration shows an application scenario of a method for handling motion delays in remote surgical operations according to an embodiment of the present disclosure.

[0051] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a master terminal 101, a master terminal host computer 102, a slave terminal host computer 103, and a slave terminal 104.

[0052] The method for handling motion delay during remote surgical operations provided in this embodiment can generally be executed by the master terminal host computer 102. The master terminal host computer 102 can acquire joint pose data from the master terminal 101 at a target frequency and send the joint pose data to the slave terminal host computer 103. The slave terminal host computer 103 can predict the delayed joint pose data based on the acquired joint pose data each time. The slave terminal host computer 103 can also send the joint pose data back to the master terminal host computer 102. The master terminal host computer 102 can evaluate the impact of the delayed remote surgical operation motion based on the joint pose data and the joint pose data before the delay, and obtain an evaluation result. The slave terminal host computer 102 can then perform weighted compensation on the delayed joint pose data based on the evaluation result to achieve compensation for the motion during remote surgical operations.

[0053] According to embodiments of this disclosure, the handheld host computer 103 can also send the acquired visual information corresponding to the patient to the main handheld host computer 102, so that the doctor can operate the main handheld computer 101 to perform remote surgery based on the acquired visual information. It should be understood that... Figure 1 The number of master terminals, master terminal host computers, slave terminal host computers, and slave terminals shown in the diagram is merely illustrative. Depending on the implementation requirements, any number of master terminals, master terminal host computers, slave terminal host computers, and slave terminals can be included.

[0054] The following will be based on Figure 1 The described scene, through Figures 2-5 The present disclosure provides a detailed description of the method for handling motion delays during remote surgical operations according to embodiments of this disclosure.

[0055] Figure 2 A flowchart illustrating a method for handling motion delays during remote surgical operations according to an embodiment of the present disclosure is shown.

[0056] like Figure 2 As shown, the method 200 includes operations S210 to S240.

[0057] During operation of S210, the joint pose data of the master end of the remote surgical robot is acquired according to the target frequency.

[0058] According to embodiments of this disclosure, a communication connection can be established between the master and slave ends of a remote surgical robot. The surgeon can manipulate the master end based on acquired visual information corresponding to the patient, and based on the communication between the master and slave ends, the slave end can perform operations consistent with those of the master end, thereby achieving remote operation of the slave end.

[0059] Because signal transmission during remote surgery can be subject to packet loss and jitter, the risk of surgical accidents increases. To address this issue, the impact of latency on remote surgical movements can be assessed based on the acquired joint pose data from the master hand, thus enabling latency compensation.

[0060] It should be noted that the visual information corresponding to the patient and the joint pose data of the main hand terminal operated by the doctor involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant countries and regions, and necessary confidentiality measures have been taken, without violating public order and good morals.

[0061] According to embodiments of this disclosure, the target frequency can characterize the set frequency for acquiring joint pose data, for example, 50Hz. The acquired joint pose data can be stored each time.

[0062] In operation S220, for each acquired joint pose data, the delayed joint pose data is predicted based on the joint pose data.

[0063] According to embodiments of this disclosure, joint pose data may include pose data of each joint on the main hand device. Based on the pose data of each joint, delayed pose data corresponding to each joint can be predicted. Specifically, the joint pose data acquired at the target frequency can represent the joint pose data before the delay, and the delayed joint pose data can represent the predicted joint pose data after a time interval of a current delay size, where the delay size corresponds to the acquired joint pose data.

[0064] During operation S230, the impact of the delay on the movement of the remote surgical procedure is evaluated based on the joint pose data and the joint pose data before the delay size, and the evaluation results are obtained.

[0065] According to embodiments of this disclosure, the joint pose data before the delay size can characterize the stored joint pose data before the delay size time interval of the currently acquired joint pose data.

[0066] During operation of S240, based on the evaluation results, weighted compensation is applied to the delayed joint pose data to compensate for the motion during remote surgical operations.

[0067] According to embodiments of this disclosure, the weights of the predicted delayed joint pose data are adjusted based on the evaluation results to make the predicted joint pose data more accurate and to compensate for the delay in the movement of the remote surgical operation.

[0068] Among them, weighted compensation can represent the adjustment of the weight coefficients of the predicted delayed joint pose data.

[0069] According to embodiments of this disclosure, for each acquired joint pose data of the master hand of the remote surgical robot, delayed joint pose data is predicted based on the joint pose data. The impact of the delay on the remote surgical operation can be evaluated based on the joint pose data and the joint pose data before the delay, and an evaluation result is obtained. This allows for real-time evaluation of the impact of network latency on the remote surgical operation, identification of potential risks, assessment of master-slave consistency between the master and slave hands, and weighted compensation of the delayed joint pose data based on the evaluation result to compensate for the remote surgical operation, thereby reducing the impact of network latency and improving surgical outcomes.

[0070] According to embodiments of this disclosure, for each acquired joint pose data, predicting delayed joint pose data based on the joint pose data includes: using a delay model equation to predict the delay magnitude of the joint pose data to obtain a predicted delay magnitude; and using a neural network to predict the delayed joint pose data based on the predicted delay magnitude and the joint pose data.

[0071] According to embodiments of this disclosure, the delay model equation can characterize the relationship between network delay and both time and non-time factors. To make the predicted delayed joint pose data more accurate, the delay magnitude itself needs to be predicted. The delay model equation can be used to predict the delay magnitude of the joint pose data, obtaining the predicted delay magnitude; then, the predicted delay magnitude and the original joint pose data are input into the neural network to output the predicted delayed joint pose data.

[0072] Currently, there is some research both domestically and internationally on the factors causing network latency through forward analysis and reverse analysis based on experimental data. However, the analysis of different network transmission protocols and different network transmission processes often has its own particularities.

[0073] For remote surgical robots, employing P2P communication and socket hole punching can eliminate the impact of public network space on servers, ensuring the required data transmission latency during remote surgery. Network simulation software can be used to analyze the main factors and their magnitude affecting P2P transmission network latency, and combined with actual measured data, a P2P network latency model can be further derived.

[0074] According to embodiments of this disclosure, based on the delay model equation, the impact of network latency on surgical movements can be calculated and evaluated in real time by combining the joint pose data and signal transmission and reception time of the master hand terminal acquired in real time, and network latency compensation can be performed based on real-time prediction.

[0075] According to embodiments of this disclosure, the delay model equation is used to predict the delay magnitude first, so that the predicted delayed joint pose data is more accurate.

[0076] According to embodiments of this disclosure, the impact of delay on remote surgical operation motion is evaluated based on joint pose data and joint pose data before the delay size, and an evaluation result is obtained. This includes: calculating a motion evaluation index based on the joint pose data and joint pose data before the delay size; and evaluating the impact of delay on remote surgical operation motion based on the motion evaluation index, and obtaining an evaluation result.

[0077] According to embodiments of this disclosure, a motion evaluation index corresponding to the joint pose data before the delay can be calculated based on the joint pose data before the delay; a motion evaluation index corresponding to the joint pose data after the delay can also be calculated based on the joint pose data after the delay. The motion evaluation index may include both the motion evaluation index before and after the delay; the motion evaluation index may include the velocity, acceleration, angular velocity, and angular acceleration of each joint on the master end of the remote surgical robot.

[0078] According to embodiments of this disclosure, the impact of network latency on remote surgical operations can be evaluated based on motion evaluation indicators before and after the latency, in terms of the degree of motion offset and the degree of motion path distortion, to obtain evaluation results.

[0079] According to embodiments of this disclosure, the impact of latency on remote surgical operation movements is evaluated based on joint pose data and joint pose data before the latency magnitude, in order to obtain evaluation results. This enables real-time evaluation of the impact of network latency on remote surgical operation movements, in order to identify potential risks and assess the master-slave consistency between the master and slave ends.

[0080] According to embodiments of this disclosure, a motion evaluation index is calculated based on joint pose data and joint pose data before the delay size, including: smoothing each acquired joint pose data and the joint pose data before the delay size corresponding to the joint pose data to obtain first joint pose data and second pose data; and performing data processing on the first joint pose data and the second joint pose data to obtain a motion evaluation index before the delay corresponding to the first joint pose data and a motion evaluation index after the delay corresponding to the second joint pose data.

[0081] According to embodiments of this disclosure, motion evaluation metrics may include motion evaluation metrics before and after the delay. The first joint pose data and the second joint pose data may include the pose data of each joint of the master hand after smoothing. The master hand of the remote surgical robot may include three joints. The joint pose data can represent the data after the delay, and the joint pose data before the delay can represent the data before the delay.

[0082] According to embodiments of this disclosure, a five-point cubic smoothing method can be used to smooth the joint pose data and the joint pose data before the delay size corresponding to the joint pose data acquired each time, so as to obtain the first joint pose data and the second joint pose data.

[0083] Since the joint pose data obtained after smoothing may not be accurate, the smoothing effect needs to be adjusted to obtain relatively accurate first and second joint pose data.

[0084] According to embodiments of this disclosure, the first joint pose data and the second joint pose data are processed separately. Specifically, the pose data of each joint can be converted into the position (mm) and angle (°) of each joint. After difference processing of the obtained data, the first and second derivatives are then performed on the data to obtain the first and second derivatives of the data, namely, velocity (mm / s) and acceleration (mm / s²). 2 ), angular velocity (° / s) and angular acceleration (° / s) 2 ).

[0085] Among them, motion assessment indicators can include the position, velocity, angular velocity, angle, angular velocity, and angular acceleration of each joint.

[0086] Taking speed as an example, difference processing can be used to represent subtracting the previous speed from the current speed.

[0087] According to embodiments of this disclosure, smoothing and interpolation processing of joint pose data are performed to further reduce the impact of hand tremors on higher-order data, thereby making the obtained motion evaluation indicators more accurate.

[0088] According to embodiments of this disclosure, the impact of time delay on the movement of remote surgical operations is evaluated based on motion evaluation indicators to obtain evaluation results, including: calculating the difference between each indicator before and after the time delay based on the motion evaluation indicators before and after the time delay to obtain a first evaluation result; obtaining the time series before and after the time delay for each indicator; and calculating the similarity between the time series before and after the time delay to obtain a second evaluation result.

[0089] According to embodiments of this disclosure, the evaluation results may include a first evaluation result and a second evaluation result; the time series may include indicators of the target quantity.

[0090] According to embodiments of this disclosure, the difference between each indicator before and after the delay can be calculated based on the motion evaluation indicators before and after the delay to obtain a first evaluation result. The first evaluation result can be used to characterize the overall deviation of the delay on the motion of remote surgical operations.

[0091] Taking speed as an example, the difference between the speed before and after the delay can be calculated to obtain the difference in speed before and after the delay in the first evaluation result.

[0092] According to embodiments of this disclosure, the DTW (Dynamic Time Warping) algorithm can be used to calculate the similarity of the time series of a target number of sampling points in each calculation cycle in real time. Each calculation cycle can characterize the evaluation and compensation calculation process performed after acquiring the joint pose data from the master hand.

[0093] According to embodiments of this disclosure, for each index involved in the motion evaluation metrics, the time series before and after the delay can be obtained first. Then, the DTW algorithm is used to calculate the similarity between the time series before and after the delay to obtain a second evaluation result. This second evaluation result can be used to characterize the impact of the delay on the path distortion of motion during remote surgical operations.

[0094] According to embodiments of this disclosure, the target quantity index may include an index corresponding to a certain number of joint pose data obtained by counting backwards from the currently acquired joint pose data.

[0095] For example, with a target quantity of 8, the joint pose of the master hand is acquired every 20ms. For a time series composed of velocity indices, where the joint pose is acquired at 180ms, the target quantity indices could include the velocity indices corresponding to the joint pose data acquired at 180ms, 160ms, 140ms, 120ms, 100ms, 80ms, 60ms, and 40ms. That is, the time series could include the velocities at 180ms, 160ms, 140ms, 120ms, 100ms, 80ms, 60ms, and 40ms. However, due to jitter caused by delay, even with a strict 20ms transmission interval, the received sequence interval is not exactly 20ms but rather a fluctuating value. Therefore, the above example is performed without considering delay jitter.

[0096] According to embodiments of this disclosure, based on motion evaluation indicators before and after the delay, the difference between each indicator before and after the delay is calculated to obtain a first evaluation result, so as to obtain the impact of the delay on the overall deviation of the motion; the similarity between the time series before and after the delay is calculated to obtain a second evaluation result, so as to obtain the impact of the delay on the motion path distortion, which facilitates more accurate motion compensation for the delay in the future.

[0097] According to embodiments of this disclosure, weighted compensation is applied to the delayed joint pose data based on the evaluation results to compensate for the motion of remote surgical operations. This includes: determining the weights corresponding to the delayed joint pose data based on the comprehensive evaluation results; and obtaining the target joint pose data based on the delayed joint pose data and the weights.

[0098] According to embodiments of this disclosure, the comprehensive evaluation result may include the evaluation result itself, and may also include an evaluation of the actual joint pose data and the predicted delayed joint pose data. After predicting the delayed joint pose data corresponding to the current joint pose data, the actual joint pose data after the time interval of the current joint pose data can be obtained after a certain period of time.

[0099] According to embodiments of this disclosure, the effect of the predicted delayed joint pose data can be evaluated based on the difference between the actual joint pose data and the predicted delayed joint pose data, as well as the first evaluation result and the second evaluation result, namely the difference of each index before and after the delay and the DTW similarity, to assess whether the effect of the predicted delayed joint pose data is positive or negative, that is, whether the effect of the predicted delayed joint pose data is improved compared with the actual joint pose data, so as to evaluate whether the prediction is accurate.

[0100] According to embodiments of this disclosure, if the prediction does not meet the expected requirements or some movements are not suitable for prediction, the weights of the delayed joint pose data obtained from the prediction can be adjusted to ensure the safety of the prediction.

[0101] According to embodiments of this disclosure, when the prediction is expected to have a positive effect and the effect is good, the weight can be increased to ensure that the situation remains relatively positive. The effect of the prediction can be determined based on the magnitude of the deviation between prediction and no prediction.

[0102] According to embodiments of this disclosure, when the magnitude of the deviation decreases, the weighting coefficients of the predicted delayed joint pose data can be adjusted to increase the weighting coefficients; when the magnitude of the deviation increases, the weighting coefficients of the predicted delayed joint pose data can be adjusted to decrease the weighting coefficients.

[0103] According to embodiments of this disclosure, when a positive effect is predicted and the effect is good, the weight of the predicted value is increased to ensure that it remains in a relatively positive state; when a positive effect is predicted but the effect is poor, the weight of the predicted value is reduced when the predicted value and the unpredicted value are fused; when a prediction has reached the point where there is no positive effect, the weight can be reduced to 0, that is, no prediction is made.

[0104] Among them, the target joint pose data can represent the pose data after adjusting the predicted delayed joint pose data according to the weights.

[0105] According to embodiments of this disclosure, the predicted delayed joint pose data can be weighted and fused at a certain ratio to obtain target joint pose data, which can then be sent to the host computer on the hand end to achieve delay compensation, reduce the impact of network latency, and improve surgical results.

[0106] According to embodiments of this disclosure, the above-described method for processing motion delay in remote surgical operations further includes: for each acquired joint pose data, calculating the delay magnitude corresponding to the joint pose data based on the signal transmission time and signal reception time corresponding to the joint pose data; calculating the delay jitter magnitude corresponding to the joint pose data based on the delay standard deviation determined by a target number of joint pose data and the delay magnitude corresponding to the joint pose data; matching the joint pose data to obtain a matching result; and determining the number of packet losses corresponding to the joint pose data based on the matching result.

[0107] According to embodiments of this disclosure, the signal transmission time can characterize the time for transmitting joint pose data, the signal reception time can characterize the time for the host computer to acquire the joint pose data, and the delay can characterize the difference between the signal reception time and the signal transmission time. The joint pose data can be transmitted in the form of data packets.

[0108] For example, in the case of data packet transmission, there is a transmission timestamp, and in the case of data packet reception, there is a reception timestamp. The delay size = reception timestamp - transmission timestamp.

[0109] According to embodiments of this disclosure, the target number of joint pose data may include a certain number of joint pose data obtained by counting backwards from the currently acquired joint pose data.

[0110] For example, with a target number of 8, the joint pose of the main hand is acquired every 20ms. In this case, the joint pose is acquired at 200ms. Therefore, the joint pose data for the target number of hands could include data acquired at 200ms, 180ms, 160ms, 140ms, 120ms, 100ms, 80ms, and 60ms. However, due to latency jitter, even if the transmission time is strictly spaced at 20ms, the received time interval will not be exactly 20ms but will be a fluctuating value. Therefore, the above example is performed without considering latency jitter.

[0111] According to embodiments of this disclosure, the moment of acquiring joint pose data from the main hand can be represented by sampling points, and a target number of sampling points can characterize the most recent sampling points from the moment the joint pose data was acquired. The standard deviation of the delay can be calculated based on the delay magnitude of each of the target number of joint pose data points; then, the quotient of this standard deviation and the delay magnitude is determined as the delay jitter magnitude.

[0112] According to embodiments of this disclosure, packet loss information of remote surgical transmission signals can be obtained through a judgment statement. Packet loss information can include the number of packet losses in the real-time display window and the total number of packet losses in real time. The number of packet losses in the real-time display window can represent the number of packet losses in the data packets of the transmitted joint pose data obtained this time. The total number of packet losses in real time can represent the total number of packet losses from the initial data packets to the data packets transmitted this time.

[0113] According to an embodiment of this disclosure, the host computer can acquire data packets transmitting joint pose data, extract the data from the data packets, and then match the extracted data according to a judgment statement to obtain a matching result. Based on the matching result, the number of packet losses corresponding to the acquired joint pose data can be determined.

[0114] According to embodiments of this disclosure, if the matching result indicates that the required data has been matched, it can be indicated that no packet loss has occurred; if the matching result indicates that the required data has not been matched, it can be indicated that packet loss has occurred, that is, the data packet is corrupted and some data has been lost.

[0115] According to embodiments of this disclosure, the lost data can also be processed by inferring the size of the lost data based on previously acquired data, allowing for the replenishment of lost data within a certain range. Before replenishing the inferred data, it is necessary to determine whether the replenished data is reasonable. If the replenished data is within a reasonable threshold, the inferred data can be directly processed for supplementary reporting. The data packets after supplementary reporting can be excluded from packet loss statistics.

[0116] Among them, the number of packet losses in the real-time display window, the total number of real-time packet losses, the delay size, and the delay jitter size can be used to characterize the evaluation indicators at the time level.

[0117] According to the embodiments of this disclosure, an initialization section is also required, namely, initialization processing statements that comprehensively consider the initial requirements of the display window, the requirements during the delay period of the first batch of data, and the requirements of the sampling points required during the calculation process at program startup. This is because, initially, not enough data is obtained, but some steps require some sampling points before analysis, such as the five-point cubic smoothing method and interpolation, both of which require a target number of sampling points. Taking 8 sampling points as an example, there are not 8 sampling points during the initial data transmission, so it is necessary to analyze and process the initial portion of the data.

[0118] In the event of packet loss, assuming that there is packet loss in the first few transmitted data packets, the following can be done: If there is packet loss in the first transmitted data packet, the data packet cannot be identified. However, since there is no data packet to be obtained before this, the data packet can be skipped.

[0119] According to embodiments of this disclosure, each time a data packet is received, the data in the data packet can be stored in a structure, such as a database, so that subsequent analyses can call the data stored in the previous structure to ensure that the most recently stored data is obtained and used for calculations.

[0120] According to embodiments of this disclosure, the impact of network latency on remote surgical procedures can also be assessed to some extent based on packet loss, latency magnitude, and latency jitter magnitude.

[0121] Figure 3 The illustration shows a schematic diagram of an evaluation prediction compensation procedure in a remote surgical robot system according to an embodiment of the present disclosure.

[0122] like Figure 3 As shown, this embodiment 300 includes a slave host computer 301, a master host computer 302, a master terminal 303, a master terminal controller 304, a slave terminal controller 305, and a slave terminal 306.

[0123] According to embodiments of this disclosure, during remote surgical operations in a remote network environment, visual information corresponding to the patient can be obtained from the handheld host computer 301 and transmitted to the main handheld host computer 302, so that the doctor can operate the main handheld terminal 303 to perform surgical operations based on the visual information displayed on the main handheld host computer 302.

[0124] According to the embodiments of this disclosure, the host computer 302 can acquire the joint pose data of the host terminal 303 at a target frequency through the host controller 304. That is, the host controller 304 can acquire the joint pose data of the host terminal 303 at a target frequency and send the host information to the host computer 302. The host information may include joint pose data and signal transmission time.

[0125] According to an embodiment of this disclosure, the master hand-held host computer 302 can transmit the acquired joint pose data to the slave hand-held host computer 301, so that the slave hand-held host computer 301 can perform the above-mentioned operations S220 and S240 according to the acquired joint pose data and signal transmission and reception time, and send the data packet containing joint pose data, delay size and other data back to the master hand-held host computer 302, so that the master hand-held host computer 302 can perform the above-mentioned operation S230.

[0126] According to embodiments of this disclosure, the evaluation and prediction compensation program may include a delay-to-motion impact evaluation program and a motion prediction compensation program. Specifically, the delay-to-motion impact evaluation program of the master handheld host computer 302 can execute the aforementioned operation S230, and the motion prediction compensation program of the slave handheld host computer 301 can execute the aforementioned operations S220 and S240.

[0127] According to embodiments of this disclosure, both the delay-based motion impact assessment procedure and the motion prediction compensation procedure can characterize the Qt program.

[0128] According to embodiments of this disclosure, the motion evaluation index, the first evaluation result, the second evaluation result, and the time-level evaluation index obtained by performing the above operation S230 using the delay-based motion impact evaluation procedure can be displayed in real time on the display interface of the host computer 302.

[0129] Since the master handheld host computer 302 has a small display workload, the prediction process can be moved to the slave handheld host computer 301 to reduce its workload. Furthermore, when actually acquiring the delay, relying solely on the delay model equation is insufficient; a relatively accurate and real-time delay value is necessary. The better the real-time performance of the delay value, the easier it is to predict the delay magnitude using the delay model equation, resulting in higher accuracy. This also makes predicting motion (delayed joint pose data) easier and more accurate. Because there is a small fluctuation in the time difference between the master handheld host computer 302 and the slave handheld host computer 301, data feedback is required to obtain a more accurate delay value. However, feedback from both the master and slave handheld host computers 302 and 301 introduces a round-trip delay, meaning the measured delay is not real-time and has three unidirectional delay lags. If prediction is performed on the slave handheld host computer, these unidirectional delays can be compensated for.

[0130] Figure 4 The illustration shows a screenshot of the dynamic window operation of the display interface of the host computer according to an embodiment of the present disclosure.

[0131] like Figure 4 The screenshots shown include the difference between p1 before and after the delay, the difference between p1dtw before and after the delay, the distance of the linear joint movement speed dtw before and after the real-time delay, the comparison of data before and after the p1 delay, the magnitude of the real-time delay jitter, and the total number of real-time packet losses.

[0132] According to embodiments of this disclosure, when there are three joints at the main hand end, these three joints can be P1 (linear joint count), P2 (rotational joint angle count), and P3 (end-effector bending angle count). The linear joint position (mm), rotational joint angle (°), and end-effector bending angle (°) can be calculated from the values ​​of P1, P2, and P3, respectively.

[0133] Figure 5 A schematic diagram illustrating the time sequence before and after a delay according to an embodiment of the present disclosure is shown.

[0134] like Figure 5 As shown, the process, path, and Euclidean distance of the velocity dtw before and after the delay of the given cycloidal motion are visualized (only the dtw process is shown, not the actual data acquired by the robot). From left to right, they are the dtw adjustment path and the corresponding two original time series, the magnitude of the index before and after the delay on the same coordinate axis, and the magnitude of the index before and after the delay on the same coordinate axis after dtw normalization.

[0135] According to embodiments of this disclosure, when the computational burden of the Qt program in the host computer 302 is not high, taking 50Hz as an example, the image can be refreshed once per calculation. However, if the calculation frequency of the Qt program is too high or computational resources need to be saved, it is necessary to consider reducing the image output frequency. That is, a method of refreshing the image after multiple calculations can be adopted. The number of calculation cycles corresponding to each image refresh can be dynamically determined according to the real-time acquired signal time. In other words, the image update frequency of the display interface of the host computer 302 can be determined according to the calculation speed, where the calculation speed can represent the calculation speed of performing the above operations S210 to S240.

[0136] According to embodiments of this disclosure, the image displayed on the display interface of the host computer 302 can be selected as needed.

[0137] In terms of real-time performance, the system generally adopts a method of calculating while timing and compensating with pause commands. The real-time time of each part of the calculation is calculated by reading the local precise time before and after the command. That is, a local time is read before the calculation and another local time is read after the calculation. The difference between the two can be used to obtain the real-time time. The difference is calculated with the difference between the real-time time interval of the signal between two predetermined image refreshes. An adjustable factor affected by factors such as the calculation frequency is added to the difference and compensated in the form of pause duration in the loop to achieve real-time display.

[0138] According to embodiments of this disclosure, the functions of the evaluation prediction compensation program may include: real-time display of all indicators and evaluation results as needed, specifically including automatically adjusting parameters according to the sampling frequency and calculating and selecting a suitable evaluation calculation frequency; automatically recording the total number of graphs to be output according to the output requirements and matching the corresponding optimal display refresh rate; automatically adjusting the window display format and size; displaying a specified number of recent data on the horizontal time axis as needed, i.e., arbitrarily adjusting the real-time display range; real-time display of the time taken for each refresh or each calculation; and arbitrarily adjusting the data range used for calculation specified by each algorithm.

[0139] According to an embodiment of this disclosure, after obtaining the target joint pose data, the host computer 301 can send the target joint pose data to the slave controller 305, so that the slave controller 305 can process the obtained target joint pose data and send it to the slave terminal 306, so that the slave terminal 306 can perform surgical operations on the patient based on the target joint pose data.

[0140] The method for evaluating and compensating for motion during remote surgical operations proposed in this invention is more suitable for open-loop systems. That is, it can perform a certain degree of compensation while ensuring safety, even when only information from the master end can be obtained, without needing to obtain information from both the master and slave ends in real time.

[0141] Based on the above-described method for handling motion delays in remote surgical operations, this disclosure also provides a device for handling motion delays in remote surgical operations. The following will be combined with... Figure 6 The device is described in detail.

[0142] Figure 6 A schematic block diagram of a device for processing motion delays in remote surgical operations according to an embodiment of the present disclosure is shown.

[0143] like Figure 6 As shown, the motion delay processing device 600 for remote surgical operations in this embodiment includes an acquisition module 610, a prediction module 620, an evaluation module 630, and a compensation module 640.

[0144] The acquisition module 610 is used to acquire joint pose data of the master end of the remote surgical robot according to the target frequency. In one embodiment, the acquisition module 610 can be used to perform the operation S210 described above, which will not be repeated here.

[0145] The prediction module 620 is used to predict delayed joint pose data based on the joint pose data acquired each time. In one embodiment, the prediction module 620 can be used to perform the operation S220 described above, which will not be repeated here.

[0146] The evaluation module 630 is used to evaluate the impact of the delay on the movement during remote surgical operations based on the joint pose data and the joint pose data before the delay size, and obtain the evaluation result. In one embodiment, the evaluation module 630 can be used to perform the operation S230 described above, which will not be repeated here.

[0147] The compensation module 640 is used to perform weighted compensation on the delayed joint pose data based on the evaluation results, so as to compensate for the motion of remote surgical operations. In one embodiment, the compensation module 640 can be used to perform the operation S240 described above, which will not be repeated here.

[0148] According to embodiments of this disclosure, the prediction module 620 includes a first prediction unit and a second prediction unit.

[0149] The first prediction unit is used to predict the delay magnitude of joint pose data using the delay model equation, and obtain the predicted delay magnitude.

[0150] The second prediction unit is used to predict the delayed joint pose data based on the predicted delay size and joint pose data using a neural network.

[0151] According to embodiments of this disclosure, the evaluation module 630 includes a calculation unit and an evaluation unit.

[0152] The calculation unit is used to calculate motion evaluation indicators based on joint pose data and joint pose data before the delay.

[0153] The evaluation unit is used to assess the impact of time delay on the movement of remote surgical procedures based on motion assessment indicators, and to obtain the evaluation results.

[0154] According to embodiments of this disclosure, the computing unit includes a first obtaining subunit and a second obtaining subunit.

[0155] The first acquisition subunit is used to smooth the joint pose data acquired each time and the joint pose data before the delay size corresponding to the joint pose data, respectively, to obtain the first joint pose data and the second pose data.

[0156] The second acquisition subunit is used to process the first joint pose data and the second joint pose data respectively to obtain the motion evaluation index before the delay corresponding to the first joint pose data and the motion evaluation index after the delay corresponding to the second joint pose data.

[0157] According to embodiments of this disclosure, the evaluation unit includes a third obtaining subunit, an acquisition subunit, and a fourth obtaining subunit.

[0158] The third obtaining subunit is used to calculate the difference between each indicator before and after the delay based on the motion evaluation indicators before and after the delay, and obtain the first evaluation result.

[0159] The sub-unit is used to obtain the time series before and after the delay for each metric, where the time series includes the target number of metrics.

[0160] The fourth sub-unit is used to calculate the similarity between the time series before and after the delay, and to obtain the second evaluation result.

[0161] According to embodiments of this disclosure, the compensation module 640 includes a determining unit and an obtaining unit.

[0162] The determination unit is used to determine the weights corresponding to the delayed joint pose data based on the comprehensive evaluation results, wherein the comprehensive evaluation results include the evaluation results.

[0163] The acquisition unit is used to obtain the target joint pose data based on the delayed joint pose data and weights.

[0164] According to embodiments of this disclosure, the processing device 600 for motion delay during remote surgical operations further includes a first calculation module, a second calculation module, a matching module, and a determination module.

[0165] The first calculation module is used to calculate the delay corresponding to the joint pose data for each acquired joint pose data, based on the signal transmission time and signal reception time corresponding to the joint pose data.

[0166] The second calculation module is used to calculate the delay jitter corresponding to the joint pose data based on the delay standard deviation determined by the target number of joint pose data and the delay magnitude corresponding to the joint pose data.

[0167] The matching module is used to match joint pose data and obtain matching results.

[0168] The determination module is used to determine the number of packet losses corresponding to the joint pose data based on the matching results.

[0169] According to embodiments of this disclosure, any plurality of modules among the acquisition module 610, prediction module 620, evaluation module 630, and compensation module 640 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the acquisition module 610, prediction module 620, evaluation module 630, and compensation module 640 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the acquisition module 610, prediction module 620, evaluation module 630, and compensation module 640 may be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0170] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a method for processing motion delays in remote surgical operations according to an embodiment of the present disclosure.

[0171] like Figure 7 As shown, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0172] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0173] According to embodiments of this disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the input / output (I / O) interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0174] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0175] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 702 and / or RAM 703 and / or one or more memories other than ROM 702 and RAM 703 described above.

[0176] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this disclosure.

[0177] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0178] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0179] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0180] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0181] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0182] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0183] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for handling motion delays during remote surgical operations, comprising: Acquire joint pose data of the master end of the remote surgical robot at the target frequency; For each acquired joint pose data, delayed joint pose data is predicted based on the joint pose data. Based on the joint pose data and the joint pose data before the delay, the impact of the delay on the movement of the remote surgical operation is evaluated to obtain the evaluation result, including: calculating the movement evaluation index based on the joint pose data and the joint pose data before the delay; evaluating the impact of the delay on the movement of the remote surgical operation based on the movement evaluation index, and obtaining the evaluation result. Based on the evaluation results, weighted compensation is applied to the delayed joint pose data to compensate for the movement of the remote surgical operation. This includes: determining the weights corresponding to the delayed joint pose data based on the comprehensive evaluation results, wherein the comprehensive evaluation results include the evaluation results; and obtaining target joint pose data based on the delayed joint pose data and the weights. The method further includes: for each acquired joint pose data, calculating the delay magnitude corresponding to the joint pose data based on the signal transmission time and signal reception time corresponding to the joint pose data; calculating the delay jitter magnitude corresponding to the joint pose data based on the delay standard deviation determined by a target number of joint pose data and the delay magnitude corresponding to the joint pose data; matching the joint pose data to obtain a matching result; and determining the number of packet losses corresponding to the joint pose data based on the matching result.

2. The method according to claim 1, wherein, The step of predicting delayed joint pose data based on the acquired joint pose data for each time includes: The delay magnitude of the joint pose data is predicted using a delay model equation, resulting in a predicted delay magnitude; and Using a neural network, the delayed joint pose data is predicted based on the predicted delay size and the joint pose data.

3. The method according to claim 1, wherein, The motion assessment metrics include motion assessment metrics before and after the time delay. The motion assessment metrics are calculated based on the joint pose data and the joint pose data before the time delay, including: The acquired joint pose data and the joint pose data before the corresponding delay size are smoothed to obtain first joint pose data and second joint pose data; and The first joint pose data and the second joint pose data are processed respectively to obtain the motion evaluation index before the delay corresponding to the first joint pose data and the motion evaluation index after the delay corresponding to the second joint pose data.

4. The method according to claim 3, wherein, The evaluation results include a first evaluation result and a second evaluation result. The evaluation of the impact of the delay on the remote surgical procedure movement based on the motion evaluation index, to obtain the evaluation results, includes: Based on the motion evaluation index before the delay and the motion evaluation index after the delay, the difference between each index before and after the delay is calculated to obtain the first evaluation result; For each metric, obtain the time series before and after the delay, wherein the time series includes the target number of metrics; and The similarity between the time series before the delay and the time series after the delay is calculated to obtain the second evaluation result.

5. A device for processing motion delays during remote surgical operations, comprising: The acquisition module is used to acquire joint pose data of the master end of the remote surgical robot according to the target frequency; The prediction module is used to predict the delayed joint pose data based on the joint pose data acquired each time. The evaluation module is used to evaluate the impact of the delay on the movement of the remote surgical operation based on the joint pose data and the joint pose data before the delay size, and obtain the evaluation result. The evaluation module includes a calculation unit and an evaluation unit. The calculation unit is used to calculate the motion evaluation index based on the joint pose data and the joint pose data before the delay size; An evaluation unit is used to evaluate the impact of the delay on the movement of the remote surgical procedure based on the motion evaluation index, and to obtain the evaluation result; The compensation module is used to perform weighted compensation on the delayed joint pose data based on the evaluation results, so as to compensate for the motion of the remote surgical operation. The compensation module includes a determining unit and an obtaining unit. A determining unit is configured to determine the weights corresponding to the delayed joint pose data based on the comprehensive evaluation results, wherein the comprehensive evaluation results include the evaluation results; The obtaining unit is used to obtain target joint pose data based on the delayed joint pose data and the weights. The device further includes: a first calculation module, a second calculation module, a matching module, and a determination module. The first calculation module is used to calculate the delay corresponding to the joint pose data for each acquired joint pose data, based on the signal transmission time and signal reception time corresponding to the joint pose data. The second calculation module is used to calculate the delay jitter size corresponding to the joint pose data based on the delay standard deviation determined by the target number of joint pose data and the delay size corresponding to the joint pose data. The matching module is used to match the joint pose data and obtain the matching result; The determination module is used to determine the number of packet losses corresponding to the joint pose data based on the matching results.

6. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 4.

8. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 4.

Images