Network System for Telemedicine and Surgical Robot System

Abstract

The present invention provides a network system for telemedicine and a surgical robot system. The network system for telemedicine includes: a relay server layer and a communication control station; the relay server layer includes a plurality of relay servers deployed between a remote end and a local end, forming multiple routing paths passing through the relay servers, and each of the routing paths passes through one of the relay servers; the communication control station is deployed between the relay server layer and the local end, and the multiple routing paths are aggregated to the communication control station and connected to the local end through the communication control station; the communication control station is configured to obtain a comprehensive score based on the reliability scores and latency scores of the multiple routing paths, and dynamically select the optimal routing path based on a preset rule according to the comprehensive scores of the multiple routing paths. With such a configuration, the reliability and stability of the routing path are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and particularly to a network system for telemedicine and a surgical robot system. Background Art

[0002] In the field of telemedicine (such as remote surgery, remote diagnosis, etc.), the currently commonly used routing methods for remote real-time communication mainly include deterministic routing, dynamic routing, and redundant routing.

[0003] The transmission path of the data packet of deterministic routing is determined before the network operation, usually based on static configuration rather than real-time network status. Dynamic routing is a network communication mechanism that optimizes the path selection in real time, and its core lies in dynamically adjusting the data transmission path according to the real-time status of the network (such as delay, bandwidth, congestion degree). Redundant routing is a network communication mechanism that transmits critical data through multi-path parallel transmission to achieve high reliability and zero-interruption fault tolerance. Its core idea is to ensure that any single-point failure will not affect the continuity of data transmission through physical or logical redundant design.

[0004] However, deterministic routing needs to ensure the status of each node on the routing path, with high costs; it cannot replace the routing path online. For example, when the network quality fluctuates, it cannot respond quickly. The routing nodes of dynamic routing are determined online, the determinacy of the routing path is low, the network quality is prone to fluctuations, and it takes time to update when the topology changes. Redundant routing only guarantees communication reliability and lacks a method to optimize latency. Summary of the Invention

[0005] The purpose of the present invention is to provide a network system for telemedicine and a surgical robot system to solve the problem that the existing routing methods for real-time communication in telemedicine are difficult to balance reliability and quickly respond to network quality fluctuations.

[0006] To solve the above technical problems, the present invention provides a network system for telemedicine, which includes: a transit server layer and a communication control station;

[0007] The transit server layer includes multiple transit servers deployed between the remote end and the local end, forming multiple routing paths passing through the transit servers, and each routing path passes through one of the transit servers;

[0008] The communication control station is deployed between the transit server layer and the local end, and multiple routing paths are aggregated to the communication control station and connected to the local end through the communication control station;

[0009] The communication control station is configured to obtain a comprehensive score based on the reliability scores and latency scores of multiple routing paths, and dynamically select the optimal routing path based on the comprehensive scores of multiple routing paths according to a preset rule.

[0010] Optionally, the reliability score is obtained by quantitatively evaluating at least one of network quality time period data, weather data, and current affairs data through deep learning using a multi-modal hybrid model.

[0011] Optionally, the multi-modal hybrid model includes: a long short-term memory network model, a convolutional neural network model, and a bidirectional encoder representation model; where

[0012] The network quality time period data is evaluated through the long short-term memory network model;

[0013] The weather data is evaluated through the convolutional neural network model;

[0014] The current affairs data is evaluated through the bidirectional encoder representation model;

[0015] The multi-modal hybrid model dynamically weights and sums the evaluation results of the long short-term memory network model, the convolutional neural network model, and the bidirectional encoder representation model respectively to obtain the reliability score.

[0016] Optionally, during the training process of the multi-modal hybrid model, the influence weights of the network quality time period data, the weather data, and the current affairs data are dynamically balanced through weighted summation, and the finally output reliability score is normalized to a percentage system.

[0017] Optionally, the latency score is obtained based on the current latency data according to a latency quantization scoring standard; the latency quantization scoring standard includes:

[0018] If the latency data is not greater than the first latency threshold, the latency score is full marks;

[0019] If the latency data is greater than the second latency threshold, the latency score is zero;

[0020] If the latency data is greater than the first latency threshold and not greater than the second latency threshold, the latency score decreases non-linearly according to the latency data, and the decreasing rate of the latency score increases with the increase of the latency data.

[0021] Optionally, the latency quantization scoring standard includes:

[0022]

[0023] Wherein, D is the delay score, t delay is the delay data, th1 is the first delay threshold, and th2 is the second delay threshold.

[0024] Optionally, the steps for obtaining the comprehensive score include:

[0025] Set a first scoring weight for the reliability score; set a second scoring weight for the delay score;

[0026] Set a short-board compensation item, which is the smaller one of the reliability score and the delay score, and set a compensation coefficient for the short-board compensation item;

[0027] The comprehensive score is the sum of the product of the reliability score and the first scoring weight, the product of the delay score and the second scoring weight, and the product of the short-board compensation item and the compensation coefficient.

[0028] Optionally, at least one of the first scoring weight, the second scoring weight, and the compensation coefficient is set adjustably according to the application scenario of telemedicine.

[0029] Optionally, based on a preset rule, the steps for dynamically selecting the optimal routing path according to the comprehensive scores of multiple routing paths include:

[0030] Before the start of telemedicine, select the routing path with the highest current comprehensive score;

[0031] After the start of telemedicine, monitor the real-time comprehensive score;

[0032] If the comprehensive score is not lower than the first scoring threshold, maintain the current routing path;

[0033] If the comprehensive score is lower than the first scoring threshold but not lower than the second scoring threshold, select the routing path with the highest comprehensive score other than the current routing path as the backup routing path, initiate a pre-connection to the backup routing path, and perform a data transmission test;

[0034] If the comprehensive score is lower than the second scoring threshold and the continuous duration reaches the duration limit, switch to the backup routing path.

[0035] To solve the above technical problems, the present invention further provides a surgical robot system, which includes a remote end, a local end, and the network system for telemedicine as described above; the remote end communicates with the local end through the network system for telemedicine.

[0036] In summary, in the remote medical network system and surgical robot system provided by the present invention, the remote medical network system includes: a relay server layer and a communication control station; the relay server layer includes a plurality of relay servers deployed between the remote end and the local end, forming multiple routing paths passing through the relay servers, and each of the routing paths passes through one of the relay servers; the communication control station is deployed between the relay server layer and the local end, and the multiple routing paths are aggregated to the communication control station and connected to the local end through the communication control station; the communication control station is configured to obtain a comprehensive score based on the reliability scores and latency scores of the multiple routing paths, and dynamically select the optimal routing path based on the comprehensive scores of the multiple routing paths according to a preset rule.

[0037] With such a configuration, on the one hand, through the setting of the relay server layer, compared with deterministic routing, it is not necessary to control each node on the routing path, which can reduce the number of determined nodes and lower the cost of the routing line; compared with dynamic routing, since the key nodes of the relay server layer are controlled, the communication quality of each path can be monitored in real time and redundant with each other, effectively improving the reliability and network communication quality. On the other hand, the selection of the routing path is based on the dual constraints of reliability and latency, improving the reliability and stability of the routing path. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Those of ordinary skill in the art will understand that the provided drawings are used to better understand the present invention and do not constitute any limitation to the scope of the present invention.

[0039] Figure 1 is a schematic diagram of a locally controlled surgical robot system.

[0040] Figure 2 is a schematic diagram of the data flow of a locally controlled surgical robot system.

[0041] Figure 3 is a schematic diagram of a remotely controlled surgical robot system.

[0042] Figure 4 is a schematic diagram of the data flow of a remotely controlled surgical robot system.

[0043] Figure 5 is a schematic diagram of the remote medical network system according to an embodiment of the present invention.

[0044] Figure 6 is a schematic diagram of the structure of a multi-modal hybrid model according to an embodiment of the present invention.

[0045] Figure 7 is a schematic diagram of the latency quantization scoring standard according to an embodiment of the present invention.

[0046] Figure 8 It is a schematic flowchart of selecting the optimal routing path based on a preset rule in an embodiment of the present invention.

[0047] Figure 9 It is a schematic diagram of the routing path switching in an embodiment of the present invention.

[0048] In the drawings: 10 - master control device; 11 - master operating arm; 12 - display device; 20 - slave robot device; 21 - robotic arm; 30 - image device; 40 - network system; 41 - relay server level; 411 - relay server; 42 - communication control station; 50 - remote end; 51 - master control device; 60 - local end; 61 - slave robot device; 62 - image device. Detailed implementation manners

[0049] To make the objectives, advantages and features of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the accompanying drawings are all in a very simplified form and not drawn to scale, only for the purpose of facilitating and clearly assisting in explaining the objectives of the embodiments of the present invention. In addition, the structures shown in the accompanying drawings are often part of the actual structures. In particular, the accompanying drawings need to show different emphases and sometimes different scales are used.

[0050] As used in the present invention, the singular forms "a", "an", "one" and "the" include plural referents, the term "or" is generally used in the sense of including "and / or", the term "several" is generally used in the sense of including "at least one", the term "at least two" is generally used in the sense of including "two or more", in addition, the terms "first", "second", "third" are used only for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first", "second", "third" may explicitly or implicitly include one or at least two of such features, "one end" and "the other end" as well as "proximal end" and "distal end" generally refer to two corresponding parts, which include not only the endpoints. In addition, as used in the present invention, "mounted", "connected", "coupled", an element "disposed" on another element should be understood in a broad sense, generally only indicating a connection, coupling, cooperation or transmission relationship between the two elements, and the two elements may be directly or indirectly connected, coupled, cooperated or transmitted through an intermediate element, and cannot be construed as indicating or implying the spatial position relationship between the two elements, that is, an element may be in any orientation such as inside, outside, above, below or on one side of another element, unless otherwise explicitly specified in the context. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. In addition, directional terms such as above, below, up, down, upward, downward, left, right, etc. are used relative to the exemplary embodiments as shown in the figures, and the upward or upper direction is towards the top of the corresponding figure, and the downward or lower direction is towards the bottom of the corresponding figure.

[0051] An object of the present invention is to provide a network system for telemedicine and a surgical robot system to solve the problem that the existing routing methods for real-time communication in telemedicine are difficult to balance reliability and quickly respond to network quality fluctuations. The following will be described with reference to the accompanying drawings.

[0052] In telemedicine (such as remote surgery, remote diagnosis, etc.), it is necessary to achieve remote real-time communication of data. Here, remote robotic surgery will be used as an example for illustration. To facilitate the understanding of remote robotic surgery, local robotic surgery will be described first. Please refer to Figure 1, which exemplarily shows a locally controlled master-slave teleoperated surgical robot system. The master-slave teleoperation here mainly refers to the master-end control device 10 and the slave-end robot device 20, which are configured in a master-slave control relationship. The operator (such as a doctor) can realize the drive control of the slave-end robot device 20 by operating the master operating arm 11 of the master control device 10 through master-slave mapping control. The slave-end robot device 20 includes a number of mechanical arms 21, which can be used to mount instruments (such as surgical instruments or other auxiliary instruments) and image acquisition devices (such as endoscopes). The movements of the master operating arm 11 driven by the operator are mapped to the movements of the instruments and image acquisition devices of the mechanical arm 21 through master-slave mapping, thereby realizing the master-slave mapping operation to perform the surgery. Further, the surgical robot system also includes an image device 30, and the image screen collected by the image acquisition device is fed back and transmitted to the image device 30, and after being processed by the image device 30, it is transmitted to the display device 12 of the master control device 10 for display for the operator to observe. The operator can operate based on the observed image screen. In application, in addition to forward mapping operation control information, the master control device 10 and the slave robot device 20 can also reversely map feedback information, such as compensating the reaction force of the contact between the instrument and the tissue to the main operating arm 11, thereby forming a closed loop from operation to feedback.

[0053] Please refer to Figure 2 , which shows the data flow of the locally controlled surgical robot system. The data flow of local robotic surgery includes two aspects, one is the motion control signal data flow, and the other is the visual signal data flow. The motion control signal data flow is: starting from the main control arm 11 of the master control device 10, the master controller of the master control device 10, after receiving the operator's action signal, converts it into the motion control instruction of the instrument and sends it to the slave controller of the slave robot device 20, and the slave controller controls the mechanical arm 21 and the mounted instrument to make corresponding actions. The visual signal data flow is that the image acquisition device installed on the mechanical arm 21 of the slave robot device 20 collects the light signal and transmits it to the image host in the image device 30, where it is converted into a visual signal and then sent to the display device 12 of the master control device 10 for display, so that the operator can observe the real-time picture of the surgical field.

[0054] After learning about the locally controlled surgical robotic system, please refer to Figure 3 , which shows a remote-controlled surgical robot system, which includes a remote terminal 50 (such as a remote operating room) and a local terminal 60 (such as a local operating room). The remote terminal 50 at least includes a master control device 51, and the master control device 51 of the remote terminal 50 is connected to the local terminal 60. Figure 1The main - end control device 10 shown may have the same or similar structures. The local end 60 at least includes a slave - end robot device 61 and an image device 62. The slave - end robot device 61 and the image device 62 of the local end 60 are connected to Figure 1 The slave - end robot device 20 and the image device 30 shown may have the same or similar structures. The remote end 50 and the local end 60 are configured to exchange data through the network system 40 to achieve remote operation.

[0055] Please refer to Figure 4 , for the data flow of the telesurgical robot system, the motion control signal is sent from the master controller of the master - end control device 51 at the remote end 50, and is transmitted through the network system 40 to the slave controller of the slave - end robot device 61 at the local end 60. The visual signal is sent from the image device 62 at the local end 60 and is transmitted through the network system 40 to the display device of the master - end control device 51 at the remote end 50. It can be seen that Figure 4 during the process of performing telesurgery, the communication quality of the network system 40 has a great impact on the surgical process. Therefore, it is necessary to optimize the network routing in real - time during the telesurgery process, reduce the latency while enhancing the reliability of the routing path, and provide guarantee for the smooth progress of the telesurgery.

[0056] Of course, telemedicine is not limited to telesurgery. It can also include tele - diagnosis, etc., which also requires one - way or two - way real - time transmission of data (such as voice data or image data, etc.) through the network system 40.

[0057] To optimize the network system for telemedicine, please refer to Figure 5 , an embodiment of the present invention provides a network system 40 for telemedicine, which includes: a relay - server layer 41 and a communication control station 42; the relay - server layer 41 includes a plurality of relay servers 411 deployed between the remote end 50 and the local end 60, forming multiple routing paths passing through the relay servers 411, and each of the routing paths passes through one of the relay servers 411; the communication control station 42 is deployed between the relay - server layer 41 and the local end 60, and the multiple routing paths are aggregated to the communication control station 42 and are connected to the local end 60 through the communication control station 42; the communication control station 42 is configured to obtain a comprehensive score according to the reliability score and the latency score of the multiple routing paths, and dynamically select the optimal routing path based on a preset rule according to the comprehensive scores of the multiple routing paths.

[0058] With such a configuration, on the one hand, through the setting of the relay server layer 41, compared with deterministic routing, it is not necessary to control each node on the routing path, which can reduce the number of nodes to be determined (only one relay server 411 needs to be controlled for each routing path), and reduce the cost of the routing line; compared with dynamic routing, since the key nodes of the relay server layer 41 (referring to the relay servers 411 on each routing path) are controlled, the communication quality of each path can be monitored in real time and redundancy can be formed among them, effectively improving the reliability and network communication quality. On the other hand, the selection of the routing path is based on the dual constraints of reliability and latency, improving the reliability and stability of the routing path.

[0059] The reliability score is described below. In an alternative exemplary embodiment, the reliability score is obtained by quantitatively evaluating at least one of network quality time period data, weather data, and current affairs data through deep learning using a multi-modal hybrid model. The inventors have found through research that other factors such as the time period, weather conditions, and politics at the location of the node (such as the relay server 411) have a certain impact on the communication quality. In applications, the time period situation at the location of the current node can be obtained by acquiring network quality time period data (such as through device logs), the weather situation at the location of the current node can be obtained through weather data (such as by scraping through a meteorological API), and the political situation at the location of the current node can be obtained through current affairs data (such as by scraping through a news API).

[0060] After collecting the network quality time period data, weather data, and current affairs data, the data can be preprocessed. The data preprocessing includes steps such as time alignment, feature engineering, and label generation.

[0061] Time alignment can unify data with different frequencies to the same time granularity.

[0062] Feature engineering may optionally include time period features, weather features, and political features, etc.; in one embodiment, the time period features include using sin / cos to encode periodic information such as hours and weeks; the weather features include associating meteorological data with the geographical location of the routing node, such as the real-time weather in the city where a data center is located. The political features include extracting the event influence weight through NLP technology, such as event sentiment analysis + entity recognition based on Bidirectional Encoder Representations from Transformers (BERT).

[0063] Label generation can use network performance metrics (such as the percentage increase in latency) as the supervision signal.

[0064] After completing the data preprocessing, a multi-modal hybrid model can be used for quantitative evaluation. Optionally, the multi-modal hybrid model includes: a long short-term memory network model (LSTM), a convolutional neural network model (CNN), and a bidirectional encoder representation model (BERT); among them, the network quality time period data is evaluated through the long short-term memory network model (LSTM), which captures periodic patterns and outputs a time period score T LSTM ; the weather data is evaluated through the convolutional neural network model (CNN), which extracts local meteorological patterns and outputs a weather score W CNN ; the current affairs data is evaluated through the bidirectional encoder representation model (BERT), which processes news events and policy documents through text analysis and outputs a political score P BERT .

[0065] Furthermore, the multi-modal hybrid model dynamically weights and sums the evaluation results of the long short-term memory network model (LSTM), the convolutional neural network model (CNN), and the bidirectional encoder representation model (BERT) respectively to obtain the reliability score R. Its calculation formula is:

[0066] R = α·T LSTM + β·W CNN + γ·P BERT

[0067] Among them, α, β, and γ are dynamic weights, and their values can be dynamically generated through the attention mechanism, for example. The structure of this multi-modal hybrid model is as Figure 6 shown. It can be understood that if one of the network quality time period data, weather data, and current affairs data does not need to participate in the quantitative evaluation, its dynamic weight can be set to zero. A demonstration example of using a multi-modal hybrid model to output a reliability score is described below.

[0068] Time period score: T LSTM = 85 (low-load period for night surgeries);

[0069] Weather score: W CNN = 40 (typhoon causing instability in the regional network);

[0070] Political score: P BERT = 70 (no major policy risks);

[0071] Dynamic weights: α = 0.5, β = 0.3, γ = 0.2 (currently typhoon season, weather weight increased)

[0072] Then the reliability score R = 0.5×85 + 0.3×40 + 0.2×70 = 68.5, and after normalization, it is 68.5 / 100. It can be seen that the reliability score of this routing path is relatively low (68.5).

[0073] Before applying the multi-modal hybrid model, it needs to be trained. Optionally, during the training process of the multi-modal hybrid model, the influence weights of the network quality time period data, the weather data, and the current affairs data are dynamically balanced through weighted summation, and the finally output reliability score is normalized to a percentage system.

[0074] In an optional example, the mean squared error (MSE) can be used as the loss function to optimize the regression accuracy of the multi-modal hybrid model for the reliability score. And L2-norm regularization can be optionally introduced to prevent the model from overfitting. In the application of telemedicine, the sample size of routing data may be limited, and L2 regularization can avoid the sensitive dependence of the model on a small amount of abnormal data (such as a certain extreme weather event). Through weighted summation, the influence weights of time period, weather, and political factors can be dynamically balanced. For example, in areas with high typhoon incidence, the weather score weight β can be increased; in policy-sensitive areas, the political score weight γ can be increased, etc. The finally output reliability score is normalized to a percentage system to unify the scoring dimension, which is convenient for weighted calculation with the delay score (also a percentage system, see the following description for details), and is convenient for application in subsequent quantitative evaluation steps. In addition, normalizing to a percentage system is conducive to intuitively displaying the reliability of the routing path (for example, 90 points can be regarded as excellent, and 60 points can be regarded as having a certain risk) to assist in making quick decisions. Of course, the loss function is not limited to the mean squared error (MSE). In some other embodiments, other loss functions such as the mean absolute error MAE, Huber loss (referring to the method of mixing MSE and MAE), or quantile loss can also be used. Those skilled in the art can select a suitable loss function according to the actual situation, or combine multiple loss functions or customize a new loss function to meet the specific task requirements.

[0075] The following describes the delay score. Telemedicine, especially application scenarios such as remote robotic surgery, is very sensitive to network latency. Therefore, a non-linear delay quantization scheme that severely reduces the score for high latency needs to be designed. In one embodiment, the delay score is obtained based on the current delay data according to the delay quantization scoring standard; the delay quantization scoring standard includes:

[0076] If the delay data is not greater than the first delay threshold, the delay score is full marks;

[0077] If the delay data is greater than the second delay threshold, the delay score is zero;

[0078] If the delay data is greater than the first delay threshold and not greater than the second delay threshold, the delay score decreases non-linearly according to the delay data, and the decreasing rate of the delay score increases as the delay data increases. Here, the first delay threshold is less than the second delay threshold.

[0079] In application scenarios such as remote robotic surgery, two different levels of delay thresholds (the first delay threshold and the second delay threshold) can be set. Delays less than the first delay threshold (e.g., 50 ms) have little impact on the surgery. Therefore, regardless of the actual value of the delay data at this time, the delay score is full marks. Delays greater than the second delay threshold (200 ms) will seriously affect the surgery. Therefore, regardless of the actual value of the delay data at this time, the delay score is zero. When the delay data is between the first delay threshold and the second delay threshold, the delay score decreases non-linearly according to the delay data, and the decreasing rate of the delay score increases as the delay data increases. Specifically, in the area close to the first delay threshold (relatively low delay area, such as 50 ms - 100 ms), the decrease in the delay score is relatively gentle, and minor delay differences have little impact on the score. In the area close to the second delay threshold (relatively high delay area, such as 150 ms - 200 ms), the score drops sharply.

[0080] In an exemplary embodiment, the delay quantization scoring criterion includes:

[0081]

[0082] where D is the delay score, t delay is the delay data, th1 is the first delay threshold, and th2 is the second delay threshold. According to this delay quantization scoring criterion, the relationship between the delay data t delay and the delay score score delay is as shown in Figure 7 It can be understood that those skilled in the art can set the first delay threshold and the second delay threshold according to the specific application scenarios of actual telemedicine. Preferably, the delay score is also normalized to a percentage system for easy unification of dimensions.

[0083] Optionally, the steps for obtaining the comprehensive score include: setting a first scoring weight for the reliability score; setting a second scoring weight for the delay score; setting a short-board compensation item, where the short-board compensation item is the smaller one of the reliability score and the delay score, and setting a compensation coefficient for the short-board compensation item; the comprehensive score is the sum of the product of the reliability score and the first scoring weight, the product of the delay score and the second scoring weight, and the product of the short-board compensation item and the compensation coefficient.

[0084] Based on the obtained reliability score R and delay score D of the routing path, the final comprehensive score of the routing path can be comprehensively analyzed. The comprehensive score faces the dual constraints of reliability and delay, and at the same time eliminates the coupling effect by introducing a short-board compensation term to avoid a single-dimensional score dominating the decision-making. The calculation formula for the comprehensive score C is:

[0085] C = w r ·R + w d ·D + μ·min(D, R)

[0086] Wherein, w r is the first scoring weight (i.e., the weight of the reliability score), w d is the second scoring weight (i.e., the weight of the delay score), and μ is the compensation coefficient of the short-board compensation term. By introducing the short-board compensation term, it can be avoided being misselected due to an overly high single-item score.

[0087] Optionally, at least one of the first scoring weight w r , the second scoring weight w d , and the compensation coefficient μ is adjustably set according to the application scenario of telemedicine. The weights in different application scenarios need to be set differently and can be dynamically adjusted according to changes in the application scenario. Preferably, the switching adjustment of the weights uses a Sigmoid function for transition (transition time ≤ 50 ms) to avoid path oscillations caused by score jumps to achieve smooth transition processing. The exemplary weight selections for some typical scenarios are shown in Table 1 below:

[0088] Table 1

[0089] Application scenario <![CDATA[w r > <![CDATA[w d > μ Weight determination principle Only unidirectional image signal transmission 0.6 0.3 0.1 <![CDATA[At this time, the motion control signal is not transmitted through the network, so the weight w of the delay score d decreases]]> Simultaneously transmit image signal and motion control instruction signal 0.4 0.5 0.1 It is necessary to simultaneously transmit motion control signal and image signal. At this time, both delay and reliability are important indicators There is a foreseeable network congestion situation 0.1 0.7 0.2 By compensating for the reliability loss, forcefully avoid congested nodes

[0090] After obtaining the comprehensive scores of multiple routing paths, it is necessary to make a decision to select a routing path based on the comprehensive scores according to a preset rule. The decision logic preferably includes a main path preference rule and a predictive switching mechanism.

[0091] Optionally, based on a preset rule, the steps of dynamically selecting the optimal routing path according to the comprehensive scores of multiple routing paths include:

[0092] Before the start of telemedicine, select the routing path with the highest current comprehensive score;

[0093] After the start of telemedicine, monitor the real-time comprehensive score;

[0094] If the comprehensive score is not lower than the first score threshold, keep the current routing path;

[0095] If the comprehensive score is lower than the first score threshold but not lower than the second score threshold, select the routing path with the highest comprehensive score outside the current routing path as the backup routing path, initiate a pre-connection to the backup routing path, and conduct a data transmission test;

[0096] If the comprehensive score is lower than the second score threshold and the continuous duration reaches the duration limit, switch to the backup routing path.

[0097] Main path preference rule: Before the start of telemedicine, sample the comprehensive scores of each routing path for a period of time and take the average value. Before the start of telemedicine, select the routing path with the highest current comprehensive score. Furthermore, during the process of telemedicine, the comprehensive scores of each routing path are updated in real time. If the network quality fluctuates during the process of telemedicine, the routing path is switched according to the predictive switching mechanism.

[0098] Predictive switching mechanism: Set two different levels of score thresholds (the first score threshold and the second score threshold, where the first score threshold is greater than the second score threshold).

[0099] Please refer to Figure 8 , in a demonstration example, the first score threshold is set to 75 and the second score threshold is set to 70. When the comprehensive score C≥75, the data communication for telemedicine can be carried out normally, and the current routing path is maintained.

[0100] When 70≤C<75, select the routing path with the highest comprehensive score outside the current routing path as the backup routing path, initiate a pre-connection to it, and conduct a data transmission test.

[0101] When C<70 and it lasts for more than the duration limit (such as 10s), force a switch to the backup routing path. The switching delay is preferably no greater than 8ms. Of course, if the comprehensive score of the current routing path returns above 75 after passing through the interval of 70≤C<75, there is no need to switch to the backup routing path, and the pre-connection and data transmission test for the backup routing path are stopped.

[0102] Optionally, if during the medical process, after the comprehensive score of the current routing path falls into the interval of 70≤C<75, if no other routing path meets the requirements (such as the comprehensive score≥75), a network quality risk is prompted.

[0103] Please refer to Figure 9 , which shows a demonstration example of routing path switching. Before the start of telemedicine, the routing path with the highest score is from the remote end 50 - the relay server 411a - the communication control station 42 - the local end 60. It is selected as the optimal routing path for data transmission.

[0104] In telemedicine, when network congestion occurs between the relay server 411a and the communication control station 42, or when a disaster weather causes damage to the network cable, the comprehensive score of its routing path will quickly drop below 70. At this time, according to the preset rules, it can be immediately switched to the optional routing path with the highest current comprehensive score, such as from the remote end 50 - relay server 411b - communication control station 42 - local end 60. Ensure that the impact of network quality fluctuations on telemedicine is minimized.

[0105] An embodiment of the present invention provides a surgical robot system, which includes a remote end 50, a local end 60, and the network system 40 of telemedicine as described above; the remote end 50 communicates with the local end 60 through the network system 40 of telemedicine. The remote end 50 and the local end 60 are configured in a master-slave control relationship. The structures and principles of other components of this surgical robot system can be understood by referring to the above and in combination with the prior art, and will not be elaborated here.

[0106] In summary, in the network system of telemedicine and the surgical robot system provided by the present invention, the network system of telemedicine includes: a relay server layer and a communication control station; the relay server layer includes multiple relay servers deployed between the remote end and the local end, forming multiple routing paths passing through the relay servers, and each of the routing paths passes through one of the relay servers; the communication control station is deployed between the relay server layer and the local end, and multiple routing paths are aggregated to the communication control station and connected to the local end through the communication control station; the communication control station is configured to obtain a comprehensive score according to the reliability score and delay score of multiple routing paths, and based on preset rules, dynamically select the optimal routing path according to the comprehensive scores of multiple routing paths. With such a configuration, on the one hand, through the setting of the relay server layer, compared with deterministic routing, it is not necessary to control each node on the routing path, which can reduce the number of determined nodes and lower the cost of the routing line; compared with dynamic routing, since the key nodes of the relay server layer are controlled, the communication quality of each path can be monitored in real time and redundancy can be formed among them, effectively improving the reliability and network communication quality. On the other hand, the selection of the routing path is based on the dual constraints of reliability and delay, improving the reliability and stability of the routing path.

[0107] It should be noted that the above-mentioned several embodiments can be combined with each other. The above description is only a description of the preferred embodiments of the present invention, and does not limit the scope of the present invention in any way. Any changes and modifications made by those of ordinary skill in the field of the present invention based on the above disclosure belong to the protection scope of the present invention.

Claims

1. A telemedicine network system, characterized in that: include: Relay server level and communication control station; The relay server level includes a plurality of relay servers deployed between the remote end and the local end, forming a plurality of routing paths through the relay servers, each of the routing paths passing through one of the relay servers; The communication control station is deployed between the relay server level and the local end, and the plurality of routing paths are aggregated to the communication control station and connected to the local end via the communication control station; The communication control station is configured to obtain a comprehensive score according to the reliability scores and delay scores of the multiple routing paths, and dynamically select the optimal routing path according to the comprehensive scores of the multiple routing paths based on preset rules; the comprehensive score is oriented to the dual constraints of the reliability score and the delay score, and at the same time introduces a short board compensation item to eliminate the coupling effect; wherein the short board compensation item is set based on the smaller one of the reliability score and the delay score.

2. The telemedicine network system according to claim 1, characterized in that: The reliability score is obtained by deep learning and using a multimodal hybrid model to quantitatively evaluate at least one of network quality period data, weather data, and current affairs data.

3. The telemedicine network system according to claim 2, characterized in that: The multimodal hybrid model includes: a long short-term memory network model, a convolutional neural network model and a bidirectional encoder representation model; wherein The network quality period data is evaluated by the long short-term memory network model; The weather data is evaluated by the convolutional neural network model; The current affairs data is evaluated by the bidirectional encoder representation model; The multimodal hybrid model dynamically weights and sums the evaluation results of the long short-term memory network model, the convolutional neural network model, and the bidirectional encoder representation model to obtain the reliability score.

4. The telemedicine network system according to claim 2, characterized in that: During the training process of the multimodal hybrid model, the influence weights of the network quality period data, the weather data and the current affairs data are dynamically balanced through weighted summation, and the reliability score finally output is normalized to a percentage system.

5. The telemedicine network system according to claim 1, characterized in that: The delay score is obtained according to the current delay data based on the delay quantification scoring standard; the delay quantification scoring standard includes: If the delay data is not greater than the first delay threshold, the delay score is full marks; If the delay data is greater than a second delay threshold, the delay score is zero; If the delay data is greater than the first delay threshold and not greater than the second delay threshold, the delay score decreases nonlinearly according to the delay data, and a decreasing rate of the delay score increases according to an increase in the delay data.

6. The telemedicine network system according to claim 5, characterized in that: The delay quantification scoring criteria include: ， Where D is the delay score, t delay is the delay data, th1 is the first delay threshold, and th2 is the second delay threshold.

7. The telemedicine network system according to claim 1, characterized in that: The steps of obtaining the comprehensive score include: Setting a first scoring weight for the reliability scoring; setting a second scoring weight for the delay scoring; The short board compensation item is the smaller one of the reliability score and the delay score, and a compensation coefficient is set for the short board compensation item; The comprehensive score is the sum of the product of the reliability score and the first score weight, the product of the delay score and the second score weight, and the product of the short board compensation item and the compensation coefficient.

8. The telemedicine network system according to claim 7, characterized in that: At least one of the first scoring weight, the second scoring weight, and the compensation coefficient may be adjustably set according to an application scenario of telemedicine.

9. The telemedicine network system according to claim 1, characterized in that: Based on the preset rules, the step of dynamically selecting the optimal routing path according to the comprehensive scores of the multiple routing paths includes: Before the telemedicine begins, selecting the routing path with the highest comprehensive score; After the telemedicine begins, monitoring the comprehensive score in real time; If the comprehensive score is not lower than the first score threshold, maintaining the current routing path; If the comprehensive score is lower than the first score threshold but not lower than the second score threshold, select the routing path with the highest comprehensive score outside the current routing path as the backup routing path, start pre-connection for the backup routing path, and perform a data transmission test; If the comprehensive score is lower than the second score threshold and the duration reaches the duration limit, switch to the backup routing path.

10. A surgical robot system, characterized in that: It comprises a remote end, a local end and a telemedicine network system according to any one of claims 1 to 9; the remote end communicates with the local end through the telemedicine network system.

Images