Remote surgery robot image transmission adaptive control method, device and equipment

Abstract

The application provides a remote surgery robot image transmission adaptive control method, device and equipment, acquires a video stream collected by an endoscope in a remote surgery process and a network state parameter of a current network link; performs parallel sensing processing on the video stream and the network state parameter, outputs surgery scene sensing information and network state sensing information; determines an adjustment scheme of an image transmission parameter based on the network state sensing information, the surgery scene sensing information and a preset surgery safety constraint condition; and performs adjustment on the image transmission parameter according to the adjustment scheme, and outputs an image transmission control signal, so that adaptive decision-making on network changes and surgery stages is realized, and the stability and safety of remote surgery image transmission in a complex network environment are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an adaptive control method, apparatus, and device for image transmission in a remote surgical robot. Background Technology

[0002] Today, remote surgical robots can perform minimally invasive surgeries by remotely controlling robotic arms, breaking geographical limitations. One of their core technological bottlenecks lies in the stable and low-latency transmission of surgical field images, while the highly dynamic nature of the network environment poses a great challenge to the stability of image transmission.

[0003] Currently, a purely rule-driven image transmission control scheme is commonly used. This scheme adjusts the image transmission parameters by periodically collecting network parameters and matching them with preset fixed parameter levels. However, its drawbacks are that the network prediction accuracy is low and the parameter weights are fixed, making it difficult to meet the stringent requirements of remote surgery for image transmission. Summary of the Invention

[0004] This application provides an adaptive control method, apparatus, and device for remote surgical robot image transmission, which enables adaptive decision-making in response to network changes and surgical stages, thereby improving the stability and security of remote surgical image transmission in complex network environments.

[0005] In a first aspect, embodiments of this application provide an adaptive control method for image transmission in a remote surgical robot, the method comprising: Acquire the video stream captured by the endoscope during remote surgery, as well as the network status parameters of the current network link; The video stream and the network state parameters are processed in parallel to generate surgical scene perception information and network state perception information. Based on the network status perception information, the surgical scene perception information, and the preset surgical safety constraints, an adjustment scheme for the image transmission parameters is determined. According to the adjustment scheme, the image transmission parameters are adjusted, and an image transmission control signal is output.

[0006] Secondly, embodiments of this application provide an adaptive control method for image transmission in a remote surgical robot, the method comprising: The data acquisition module is used to acquire the video stream captured by the endoscope during remote surgery, as well as the network status parameters of the current network link; The parallel perception processing module is used to perform parallel perception processing on the video stream and the network state parameters, and output surgical scene perception information and network state perception information. The constraint optimization decision module is used to determine the adjustment scheme of image transmission parameters based on the network state perception information, the surgical scene perception information, and preset surgical safety constraints. The transmission signal output module is used to adjust the image transmission parameters according to the adjustment scheme and output an image transmission control signal.

[0007] Thirdly, embodiments of this application also provide an electronic device, which includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the remote surgical robot image transmission adaptive control method as described in any of the embodiments of this application.

[0008] This application provides an adaptive control method, apparatus, and device for image transmission in a remote surgical robot. The method includes: acquiring a video stream captured by an endoscope during remote surgery, and network state parameters of the current network link; performing parallel sensing processing on the video stream and network state parameters to output surgical scene perception information and network state perception information; determining an adjustment scheme for image transmission parameters based on the network state perception information, surgical scene perception information, and preset surgical safety constraints; adjusting the image transmission parameters according to the adjustment scheme, outputting an image transmission control signal, and outputting surgical scene perception information and network state perception information through parallel sensing processing. The method achieves adaptive decision-making in response to network changes and surgical stages, solving the problems of low network prediction accuracy and fixed parameter weights in pure rule-driven schemes, and improving the stability and security of remote surgical image transmission in complex network environments. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the exemplary embodiments of this application, the accompanying drawings used in describing the embodiments are briefly introduced below. Obviously, the accompanying drawings described are only a portion of the embodiments to be described in this application, and not all of them. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0010] Figure 1 A flowchart illustrating an adaptive control method for image transmission in a remote surgical robot provided in an embodiment of this application; Figure 2 This is a schematic diagram of the intelligent perception-constraint optimization-smooth execution three-layer hybrid control architecture involved in the embodiments of this application; Figure 3 A schematic flowchart illustrating another adaptive control method for image transmission in a remote surgical robot provided in an embodiment of this application; Figure 4 A flowchart illustrating another adaptive control method for image transmission in a remote surgical robot provided in this application embodiment; Figure 5 A flowchart illustrating another adaptive control method for image transmission in a remote surgical robot provided in this application embodiment; Figure 6 This is a schematic diagram of the structure of an adaptive control device for image transmission of a remote surgical robot provided in an embodiment of this application; Figure 7 This is a schematic diagram of another adaptive control device for image transmission of a remote surgical robot provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0011] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present application, not the entire structure.

[0012] In this specification, numerous specific technical details are described in certain places to enable those skilled in the art to understand the complete technical solution. However, it should be understood that embodiments of this application can be implemented without these specific technical details. Such detailed descriptions of technical details should not be considered as limitations on this application, and the scope of protection of this application is defined only by the claims. Elsewhere, well-known structures, connections / positional relationships, circuits, and / or other details may not be shown in detail to avoid misunderstanding of the inventive points of this application by the general public. In this specification, accompanying drawings illustrate schematic diagrams of several embodiments of this application. However, the drawings are merely illustrative, and it should be understood that changes in mechanical structure, connections / positional relationships, physical composition, electrical aspects, and steps can be made without departing from the spirit and scope of this application. Such changes can be either substitutions or combinations of elements from several embodiments of this application, or substitutions or combinations of well-known content. The terminology used herein is only for describing particular embodiments and is not intended to limit this application. Spatial relative terms, such as “below,” “lower,” “above,” “upper,” “middle,” “center,” “inner,” “outer,” “central,” “edge,” etc., are used for ease of explanation to describe the relationship between one component or feature and another shown in the figure. It should be understood that spatial relative terms are used only under the condition of the orientation of the device in use or operation (other than the orientation specifically defined in the figure), and are not necessarily unique or constant. For example, if the device in the figure is rotated 180° up and down along the paper, then an element described as “below” other components or features will become “above” other components or features. Therefore, the exemplary term “below” can encompass both above and below directions, depending on how the device is positioned. The device can also be positioned in other directions (e.g., rotated 90° or positioned in other directions), and the spatial relative descriptive terms used herein will be interpreted accordingly.

[0013] As used herein, “several,” “one,” and “the” are intended to include plural forms as well, unless the context otherwise indicates. It should be further understood that the terms “comprising” and / or “including” specify the presence of the said feature, step, operation, element, and / or component, without excluding the presence of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0014] Finally, the terms “or” and “and / or” as used herein should be interpreted inclusively, meaning either one or any combination thereof. Therefore, “A, B, or C” or “A, B, and / or C” means any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C. Exceptions to this definition will only occur when the combination of elements, functions, steps, or actions is inherently mutually exclusive in some way.

[0015] Before introducing the technical solutions provided in the embodiments of this application, the application scenarios of the solutions will be explained first. This embodiment is applicable to various scenarios that require adaptive control of image transmission from remote surgical robots.

[0016] Currently, parameter control technology for image transmission in remote surgical robots has become a research hotspot in the industry. Existing technologies are mainly divided into two categories: one is a purely rule-driven control scheme, and the other is a purely intelligent control scheme. Both of these schemes have obvious limitations and are difficult to meet the stringent requirements of remote surgery.

[0017] Option 1: Pure Rule-Driven Image Transmission Control Scheme. The core objective of this scheme is to achieve simple adjustments to image transmission parameters through fixed rules, ensuring basic image transmission needs. The scheme mainly consists of a network parameter acquisition module, a parameter lookup module, and a smoothing execution module. The specific workflow is as follows: the network parameter acquisition module periodically collects network parameters such as available bandwidth, one-way latency, latency jitter, and packet loss rate (acquisition cycle 200-500ms), and eliminates noise through Kalman filtering or low-pass filtering; the parameter lookup module presets fixed parameter levels (e.g., different bandwidths correspond to fixed bitrates, frame rates, and resolutions), and matches the corresponding parameter level based on the instantaneous values ​​of the collected network parameters; the smoothing execution module achieves smooth parameter adjustment through simple, gradual switching logic, avoiding sudden image changes. The signal transmission relationship of this scheme is: network parameter acquisition module, filtering processing, parameter lookup module, smoothing execution module, and image transmission module, ultimately achieving the adjustment of image transmission parameters. Notably, recognition during the surgical phase relies on manual marking by the doctor or parsing of the surgical robot's robotic arm instructions, without the need for an intelligent model.

[0018] However, the pure rule-driven solution has the following drawbacks: First, the network prediction accuracy is low. The linear prediction algorithm cannot cope with complex network fluctuations such as 5G base station switching and millimeter wave blockage. It can only make decisions based on instantaneous network values ​​or simple averages, which can easily lead to insufficient bandwidth reservation due to prediction deviations, causing data queue backlog and excessive end-to-end latency. Second, the automation level of surgical stage recognition is low. Doctors need to manually mark or parse the private protocol interface of the surgical robot, which not only increases the frequency and cumulative time of doctors' non-clinical intervention through the human-computer interaction interface, but also has problems with poor interface compatibility and recognition lag. Third, it lacks the ability to predict network deterioration risks and can only passively respond to network changes. When the network status suddenly deteriorates, it can only drastically degrade parameters, resulting in screen stuttering and sudden changes, affecting surgical operations. Fourth, the parameter weights are set in a fixed manner and cannot be dynamically adjusted according to the differentiated needs of the surgical stage (such as high resolution for the suturing stage and high frame rate for the separation stage), resulting in wasted bandwidth or insufficient image quality.

[0019] Option 2: Purely Intelligent Image Transmission Control Scheme. The core objective of this scheme is to leverage the predictive capabilities of an intelligent model to improve the accuracy of parameter control. The scheme primarily consists of a network parameter acquisition module, an intelligent prediction module, and a parameter output module. The specific workflow is as follows: the network parameter acquisition module collects network parameters and surgical image data and inputs them into the intelligent model; the intelligent model outputs control commands for image transmission parameters (bitrate, frame rate, resolution, etc.); the parameter output module directly executes the intelligent model's commands to adjust the parameters. This scheme requires no complex rule constraints and relies entirely on the predictive results of the intelligent model for decision-making.

[0020] However, purely intelligent-driven solutions have the following drawbacks: First, they lack interpretability and security boundaries. The decision-making logic of the intelligent model is difficult to trace, does not meet the safety certification requirements for medical devices, and cannot be applied to clinical surgery. Second, there is a risk of failure due to data drift. When the network environment or surgical scenario changes, the intelligent model is prone to prediction deviations, and there is no effective fallback mechanism, which may lead to safety hazards such as excessive surgical delays. Third, the engineering implementation is not complete enough. It does not combine mature mechanisms such as multi-link collaboration and image quality baseline alarms, which may lead to the inability to perform surgery normally under extreme network conditions. Fourth, the model deployment cost is high. It does not adopt a lightweight design, the inference latency is difficult to meet the real-time requirements of surgery, and it requires additional high-end computing hardware, which is not conducive to porting and adaptation to devices from different manufacturers.

[0021] In summary, while rule-driven solutions are widely used, they struggle to cope with dynamic network fluctuations and varying needs at different surgical stages, easily leading to stuttering, latency, or insufficient image quality. Pure AI-driven solutions, while improving prediction accuracy, lack interpretability in their decisions and are susceptible to data drift that could cause model failure, lacking a robust safety margin. Therefore, balancing the accuracy of intelligent prediction with the security of rule-based control is a pressing technical challenge in this field.

[0022] Example 1: Figure 1 This is a flowchart illustrating an adaptive control method for image transmission of a remote surgical robot provided in an embodiment of this application. This embodiment is applicable to various scenarios that require adaptive control of image transmission of a remote surgical robot. The method can be executed by an adaptive control device for image transmission of a remote surgical robot. This device can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC, or a server.

[0023] like Figure 1 As shown, the method includes: S110. Acquire the video stream captured by the endoscope during remote surgery, as well as the network status parameters of the current network link.

[0024] The current network link refers to the real-time communication connection used during remote surgery to carry endoscopic video streams from the surgical robot to the remote doctor's console.

[0025] Network status parameters refer to various data indicators used to quantify the real-time transmission quality of the current network link. For example, network status parameters may include, but are not limited to, available bandwidth, one-way latency, latency jitter, and packet loss rate.

[0026] Specifically, during remote surgery, the video stream captured by the endoscope provides a real-time view of the surgical area, enabling remote surgeons to perform operations. This video stream needs to be transmitted in real-time from the surgical robot to the remote surgeon's control console via the current network link. To ensure that the transmission quality of the video stream on this link meets the safety requirements of the surgical procedure, it is also necessary to simultaneously acquire the network status parameters of the current network link, i.e., multiple quantitative indicators that reflect the real-time transmission quality of the link during video stream transmission.

[0027] S120: Perform parallel perception processing on the video stream and network status parameters, and output surgical scene perception information and network status perception information.

[0028] Among them, surgical scene perception information refers to the structured description information of the current surgical process stage, which is used to reflect the type and progress status of the surgical operation.

[0029] Among them, network status awareness information refers to the quantitative representation information of the current network link transmission capacity, which is used to reflect the bandwidth support level that the current network link can provide for video stream transmission.

[0030] Specifically, to comprehensively understand the changing trends of the link environment upon which video stream transmission depends and the surgical progress status reflected in the video content, parallel perceptual processing of these two types of input data is required. For the video stream, feature extraction and recognition are performed on keyframe images to determine the specific operational stage of the current surgery, forming surgical scene perception information. For network state parameters, historical changes in parameters such as available bandwidth, latency, jitter, and packet loss rate are analyzed to predict the bandwidth support capacity of the current network link in the future, forming network state perception information.

[0031] S130. Based on network status awareness information, surgical scene awareness information, and preset surgical safety constraints, determine the adjustment scheme for image transmission parameters.

[0032] Among them, the preset surgical safety constraints refer to the hard threshold standards that must be met during image transmission to ensure the safety of remote surgical operations.

[0033] Among them, image transmission parameters refer to various adjustable configuration items used to control the encoding and transmission behavior of video streams during transmission.

[0034] The adjustment scheme refers to the target combination of image transmission parameters determined for the current network link carrying the video stream transmission, based on the actual state of the current network link and under the premise of meeting the preset surgical safety constraints.

[0035] Specifically, network status awareness information reflects the bandwidth support capacity of the current network link carrying video stream transmission over a future period, determining the upper limit of the bitrate achievable in image transmission parameters. Surgical scene awareness information reflects the specific operational stage of the current surgery, determining the priority order of resolution and frame rate in the optimization objectives of image transmission parameters. Preset surgical safety constraints specify the hard boundaries that must be met for end-to-end latency, image quality baseline, buffer jitter matching, and forward error correction redundancy rate. Based on these three inputs, and ensuring that all preset surgical safety constraints are simultaneously met, the set of parameter values ​​that best adapts to the needs of the current surgical stage under the current network link status is selected from all feasible combinations of image transmission parameters. This set of values ​​constitutes the image transmission parameter adjustment scheme.

[0036] S140. According to the adjustment plan, adjust the image transmission parameters and output the image transmission control signal.

[0037] Among them, the image transmission control signal refers to the instruction information used to drive the image transmission module to perform actual parameter switching operations. Its content includes the final target values ​​of various image transmission parameters determined in the adjustment scheme and / or the allocation strategy required for multi-link coordination.

[0038] Specifically, the parameters in the adjustment scheme can be applied to the image transmission module in a smooth transition, gradually bringing the current encoding and transmission configuration closer to the target values ​​specified in the adjustment scheme, thus avoiding visual shocks caused by sudden parameter changes in the video stream. During the application of these parameters, an image transmission control signal containing the final values ​​of the adjusted image transmission parameters and the multi-link allocation strategy is generated simultaneously. This signal, as the final output of the entire adjustment process, carries all the necessary information from decision-making to execution of the adjustment scheme, driving the image transmission module to complete the actual parameter switching and video stream distribution operations. This ensures that the encoding quality, transmission bandwidth usage, and multi-link carrying method of the video stream remain consistent with the target state determined by the adjustment scheme during subsequent transmission.

[0039] This application provides an adaptive control method for image transmission in a remote surgical robot. It acquires video streams captured by an endoscope during remote surgery, along with network state parameters of the current network link. Parallel sensing processing is performed on the video streams and network state parameters to output surgical scene perception information and network state perception information. Based on the network state perception information, surgical scene perception information, and preset surgical safety constraints, an adjustment scheme for image transmission parameters is determined. According to the adjustment scheme, the image transmission parameters are adjusted, and an image transmission control signal is output. Parallel sensing processing outputs surgical scene perception information and network state perception information, and the adjustment scheme for image transmission parameters is determined based on these two types of perception information combined with preset surgical safety constraints. This achieves adaptive decision-making in response to network changes and surgical stages, solving the problems of low network prediction accuracy and fixed parameter weights in purely rule-driven schemes, and improving the stability and security of remote surgical image transmission in complex network environments.

[0040] In this embodiment, optionally, in addition to the parallel sensing processing of the video stream and network state parameters, and the output of network state sensing information and surgical scene sensing information, the method also utilizes network state parameters and current buffer occupancy to predict potential risks in network transmission. Specifically, the method further includes: Based on network status parameters and current cache occupancy, a risk prediction model is used to determine the probability of network transmission risks and obtain risk warning and perception information.

[0041] The current buffer occupancy rate refers to the proportion of data currently occupied in the buffer area of ​​the image transmission module used to smooth network jitter. The risk prediction model is a model that infers the likelihood of future network transmission delays exceeding limits based on current network state parameters and the current buffer occupancy rate.

[0042] Among them, network transmission risk probability refers to the probability value used to characterize the likelihood of future network transmission delays exceeding limits. Risk warning and perception information refers to structured output information that includes the aforementioned network transmission risk probability.

[0043] In the parallel sensing processing stage, in addition to acquiring surgical scene sensing information and network status sensing information through video streams and network status parameters, a mechanism for early prediction of network transmission risks is also introduced. This mechanism uses the current network status parameters and current buffer occupancy rate as inputs, and uses a risk prediction model to make probabilistic inferences about possible end-to-end transmission delay exceeding limits in the future, and outputs the inference results in the form of risk warning sensing information.

[0044] Accordingly, the adjustment of image transmission parameters according to the adjustment plan also includes: when the risk probability in the risk warning perception information exceeds the preset warning threshold, the bit rate in the adjustment plan is preventively reduced.

[0045] Among them, the preset early warning threshold refers to the pre-set critical value of the probability of network transmission risk that needs to be triggered to determine whether the risk has reached the level that requires preventive operation.

[0046] Bitrate refers to the amount of data generated per unit time during video stream encoding and compression. Preventative bitrate reduction refers to the proactive measure of reducing the bitrate by a certain margin before the probability of network transmission risks actually causes latency to exceed limits.

[0047] Specifically, when adjusting image transmission parameters, while completing the regular parameter switching according to the adjustment plan, the risk probability contained in the risk warning perception information will also be judged. When the risk probability exceeds the preset warning threshold, the bit rate in the adjustment plan will be proactively reduced before the delay exceeds the limit, that is, the amount of video data output per unit time will be reduced in advance, so as to reserve sufficient buffer margin for the network link.

[0048] The above solution transforms bitrate adjustment from a passive response to an active prevention by predicting risk warning information in advance. It releases bandwidth pressure before latency exceeds the limit, thereby avoiding the visual jumps and operation interruptions caused by sudden changes in network conditions that force a sharp reduction in resolution or frame rate. This makes the entire adaptive adjustment process smoother and more stable, effectively ensuring the continuity and stability of the video stream during remote surgery.

[0049] As an optional implementation, in this embodiment, surgical scene perception information can be obtained using a lightweight convolutional neural network model; network state perception information can be obtained using a lightweight temporal convolutional network model; and risk warning perception information can be obtained using a lightweight multilayer perceptron model. The training process for the above three models is as follows: First, dataset collection. Remote surgical clinical data from 10 medical institutions were collected, specifically including time-series network parameter data covering various network link types such as 5G links, wired links and satellite links with a total duration of no less than 400 hours, endoscopic video data covering various minimally invasive surgical scenarios and no less than 200 surgeries, and surgical stage annotation data annotated by senior surgeons with an accuracy rate of no less than 99%.

[0050] Secondly, data preprocessing. Kalman filtering was used to eliminate noise and normalize the network parameter data to the [0,1] interval. Then, the data was divided into training set, validation set, and test set according to the ratio of 70%, 20%, and 10%. Keyframes were extracted from the endoscopic video data, and size normalization and grayscale enhancement were performed. Blurry frames and occluded frames were removed.

[0051] Further, model training. The temporal convolutional network model uses an adaptive moment estimation optimizer with a learning rate of 0.001, a batch size of 32, and 100 training epochs, aiming to minimize the deviation between the predicted and actual bandwidth values. The convolutional neural network model employs a transfer learning strategy, fine-tuning a pre-trained and pruned lightweight network model with a stochastic gradient descent optimizer, a learning rate of 0.0005, a batch size of 16, and 50 training epochs, aiming to improve the accuracy of surgical stage identification. The multilayer perceptron model uses an adaptive moment estimation optimizer with a learning rate of 0.001, a batch size of 64, and 80 training epochs, aiming to improve the accuracy of risk prediction.

[0052] In addition, a federated learning algorithm is used to achieve collaborative training of data from multiple medical institutions. The specific process is as follows: the central server distributes the trained base model to each cooperating medical institution node; each medical institution node uses local privacy data to fine-tune the base model locally, with 20 rounds of local fine-tuning and a learning rate of 0.0005. The model weight update is calculated and differential privacy encryption is performed. During the local fine-tuning process, the video stream only resides in the memory buffer and is released immediately after the calculation is completed, without generating any persistent copies in the local storage medium; each medical institution node uploads the encrypted weight update to the central server, and the server uses the federated learning algorithm to aggregate the weight updates of all nodes to generate a globally optimized model; the central server redistributes the aggregated global model to each medical institution node, repeating the above steps iterating for 3 to 5 rounds until the performance of the global model on the test set of each medical institution tends to be stable.

[0053] During factory deployment, the surgical robot is equipped with this global model, while reserving a local fine-tuning interface: For new medical institutions without local training data, 5 to 10 postoperative surgical video data are used, with each video expanded to approximately 2000 training sample pairs through a sliding time window slicing method. During fine-tuning, the parameters of the temporal convolutional network and the feature extraction backbone network of the convolutional neural network model are frozen, and only the parameters of the final fully connected classification layer are updated. A regularization term with a weight decay regularization coefficient of 1e-4 is used to suppress overfitting. The fine-tuning effect is confirmed using a leave-one-out cross-validation strategy. If the average accuracy is less than 2% higher than the factory model, the fine-tuned parameters are abandoned and the factory weights are reverted. For medical institutions with local data, further optimization can be performed based on the global model to ensure model performance.

[0054] Through the above training and fine-tuning mechanisms, the three models can achieve multi-center collaborative optimization while protecting the data privacy of various medical institutions, ensuring that the factory-delivered models have good generalization capabilities, and at the same time providing differentiated local adaptation capabilities for different medical institutions.

[0055] This solution can be implemented using a three-layer hybrid control architecture: intelligent perception, constraint optimization, and smooth execution. Figure 2 This is a schematic diagram of the intelligent perception-constraint optimization-smooth execution three-layer hybrid control architecture of the present invention. In the intelligent perception layer, keyframes of the endoscopic video are input to a lightweight convolutional neural network, which outputs surgical stage labels; historical network parameters are input to a lightweight temporal convolutional network, which outputs bandwidth estimates; historical network parameters and current cache occupancy are input to a lightweight multilayer perceptron, which outputs the probability of network degradation risk. The constraint optimization layer receives the surgical stage labels and bandwidth estimates output from the intelligent perception layer, and uses a greedy algorithm to solve the problem by combining preset surgical safety parameter constraints. It outputs the optimal combination of image transmission parameters, including bitrate, resolution, frame rate, cache size, and forward error correction redundancy rate. Simultaneously, the constraint optimization layer determines whether the probability of network degradation risk exceeds a preset threshold; if so, it generates a warning signal. The warning signal and the optimal combination of image transmission parameters enter the smooth execution layer. The smooth execution layer progressively adjusts the parameters through a hysteresis interval module and achieves multi-link collaborative transmission through a multi-link aggregation module, ultimately outputting image transmission control parameters to the image transmission module for adjustment.

[0056] It should be noted that the surgical stage labels and network degradation risk probabilities output by the intelligent perception layer described in this paper are passed as auxiliary decision-making suggestions to the constraint optimization layer. A physical or virtual image quality lock button is always provided on the human-computer interaction interface. After the surgeon presses this button, the smooth execution layer will forcibly maintain the current resolution, frame rate, and bitrate combination, ignoring the latest solution output of the constraint optimization layer, until the surgeon manually unlocks the image. Furthermore, when the intelligent model triggers failure rollback or the control mode switches from intelligent-driven to pure rule-driven, a specific prompt tone is emitted through the surgeon's headset channel to ensure that the surgeon remains fully aware of the transfer of control over image transmission.

[0057] Example 2: Figure 3 This illustration shows an adaptive control method for image transmission in a remote surgical robot, provided in this application embodiment. Based on the foregoing embodiments, this embodiment details the specific implementation of determining surgical scene perception information and network state perception information. For specific implementation methods, please refer to the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here. Figure 3 As shown, the method specifically includes the following steps: S210. Acquire the video stream captured by the endoscope during remote surgery, as well as the network status parameters of the current network link.

[0058] S220. Based on network state parameters, determine the low-order bandwidth estimate of the network bandwidth through a time-series feature extraction model to obtain network state perception information.

[0059] Among them, the time-series feature extraction model refers to a computational model used to analyze the changes in network state parameters over time and infer the future network state accordingly. The low-order bandwidth estimate refers to an estimate representing the lower limit of the available bandwidth of the current network link in the future.

[0060] Specifically, during remote surgery, network state parameters change continuously over time. A temporal feature extraction model is used to capture the historical trends of these parameters. This model takes network state parameters from multiple past sampling points as input, analyzes the patterns of these parameters changing over time to predict the future trend of network bandwidth, and extracts a low-order bandwidth estimate—the minimum guaranteed available bandwidth in the future. This value is then output as network state awareness information. For example, optionally, the low-order bandwidth estimate can be Bp5.

[0061] In this embodiment, optionally, the specific implementation steps for determining the low-order bandwidth estimate of the network bandwidth based on network state parameters through a time-series feature extraction model may include: The first step is to obtain network state parameters for a preset number of collection cycles to form a historical network state sequence.

[0062] The preset number of sampling periods refers to a predetermined number of consecutive sampling intervals on the timeline used to collect network state parameters. The sampling period is the time interval between two consecutive collections of network state parameters, fixed at 200 milliseconds; that is, a set of network state parameters for the current network link is collected every 200 milliseconds. By fixing the sampling period, network state parameters are continuously acquired and recorded according to a uniform time rhythm. Based on this, the length of the time window covered by the historical network state sequence can be changed by adjusting the value of the preset number of sampling periods. For example, during delicate surgery, the preset number is set to 10 to 15, corresponding to a time window of 2 to 3 seconds, thus more accurately capturing the changing trends of network parameters; during routine surgery, the preset number is set to 6 to 10, corresponding to a time window of 1.2 to 2 seconds, ensuring both trend capture accuracy and real-time requirements.

[0063] Network status parameters refer to various quantitative indicators used to characterize the current network link transmission quality. These parameters include available bandwidth, one-way latency, latency jitter, and packet loss rate. Available bandwidth refers to the amount of data transmission the current network link can carry per unit time; one-way latency refers to the time taken for video stream data to be transmitted one-way from the sender to the receiver; latency jitter refers to the fluctuation in the one-way latency between adjacent data packets; and packet loss rate refers to the proportion of data packets that fail to reach the receiver within a unit of time out of all transmitted data packets.

[0064] Among them, the historical network state sequence refers to a data set composed of multiple sets of network state parameters collected continuously within a preset number of collection periods, arranged in chronological order.

[0065] Specifically, during the transmission of remote surgical video streams, network status parameters are continuously acquired at fixed acquisition intervals. When it is necessary to use historical data to predict the changing trend of the current network link, each set of network status parameters acquired within a preset number of acquisition intervals is arranged sequentially according to their corresponding time sequence to form a complete historical network status sequence.

[0066] For example, if the acquisition period is set to 200ms and the preset quantity is 10, then the 10 sets of network status parameters acquired continuously in the past 2 seconds, arranged in order from early to late time, constitute the historical network status sequence. This sequence covers the complete status change trajectory of the current network link in the recent period.

[0067] The second step involves filtering and preprocessing the historical network state sequence, then extracting the time dependency of the network state parameters using a lightweight temporal convolutional network to determine the low-order bandwidth estimate of the available bandwidth within the future time window.

[0068] Among them, lightweight temporal convolutional networks are a type of temporal neural network model with fewer parameters, suitable for running on embedded platforms, and used to extract the dependencies of various parameters in historical network state sequences over time.

[0069] The future time window refers to a preset period of time extending forward from the current moment. Available bandwidth refers to the amount of data transmission that the current network link can provide within this future time window.

[0070] Specifically, after the historical network state sequence is formed, since the original network state parameters may contain noise caused by instantaneous fluctuations in the network, the historical network state sequence is first filtered and preprocessed to eliminate these abnormal jumps, so that the change trend reflected by the sequence is more stable and reliable.

[0071] The historical network state sequence, after filtering and preprocessing, is then fed into a lightweight temporal convolutional network. This network uses its internal temporal convolutional structure to analyze the coordinated changes in available bandwidth, one-way latency, latency jitter, and packet loss rate over multiple acquisition cycles, thereby capturing the dependencies between these parameters over time. Based on the extracted temporal dependencies, the lightweight temporal convolutional network further infers the trend of available bandwidth changes within future time windows and determines a high-confidence low-bit bandwidth estimate. This estimate serves as a key basis for setting the upper limit of the bit rate in subsequent adjustment schemes.

[0072] For example, when the historical network state sequence shows that the available bandwidth has been declining continuously over the past few seconds and the latency jitter has been increasing, the lightweight temporal convolutional network infers the lower limit of the available bandwidth in the future time window based on this time dependency. This lower limit value is the low bandwidth estimate.

[0073] The above method eliminates network parameter noise through filtering preprocessing, making the time series data stable and reliable. Then, a lightweight temporal convolutional network is used to extract time dependencies, accurately inferring the low-order estimate of the future available bandwidth, providing a reliable constraint on the upper limit of the bit rate, and balancing the estimation accuracy and platform real-time performance.

[0074] As an optional implementation, in the process of acquiring network state awareness information, in addition to outputting a low-order bandwidth estimate for subsequent constraint optimization calculations through the time-series feature extraction model, the model can also simultaneously output a high-order bandwidth estimate. For example, optionally, the high-order bandwidth estimate can be Bp95.

[0075] The low-order bandwidth estimate represents the minimum guaranteed bandwidth available to the current network link over a future period, used to construct the hard constraint for the bitrate cap in subsequent adjustment schemes. The high-order bandwidth estimate, on the other hand, represents the potential upper limit of the available bandwidth for the current network link over a future period. This value is not involved in the robot's autonomous constraint optimization calculations but is provided to the remote doctor as auxiliary display information, allowing them to intuitively understand the current network's performance ceiling and helping them determine whether the current network state is trending towards stability or fluctuating drastically, thus providing a decision-making reference for whether the doctor should manually lock the image quality level. The low-order and high-order bandwidth estimates together constitute the network state perception information.

[0076] S230. Based on keyframes in the video stream, the label of the current surgical stage is determined through an image feature extraction model to obtain surgical scene perception information.

[0077] In this context, a keyframe in a video stream refers to a single frame image extracted from a continuous video stream acquired by an endoscope at preset time intervals, used to represent the content of the current scene.

[0078] The image feature extraction model refers to a computational model used to analyze the image content of keyframes in a video stream and identify the surgical procedure stage category. The current surgical stage label refers to the identification information characterizing the surgical procedure stage category to which the current keyframe belongs. Optionally, the specific categories of the current surgical stage label include separation and dissection stage, suturing and anastomosis stage, hemostasis and irrigation stage, dangerous area operation stage, and lens wiping and movement stage.

[0079] Specifically, during remote surgery, the video stream captured by the endoscope consists of continuous frames. Keyframes representing the current scene are extracted from this video stream at preset time intervals. The extracted keyframes are then fed into an image feature extraction model. This model analyzes the instrument morphology, tissue state, and relative position of the instruments and tissues presented in the endoscopic images within the keyframes to identify the current surgical stage and outputs a corresponding label for that stage.

[0080] For example, when the extracted keyframe shows that the electric hook is contacting the tissue surface and smoke is being generated, the image feature extraction model outputs a label representing the hemostasis and irrigation stages based on the features of the image. This label serves as the content of the surgical scene perception information.

[0081] In this embodiment, optionally, the specific implementation steps of determining the current surgical stage label based on keyframes in the video stream using an image feature extraction model may include: The first step is to extract keyframes from the video stream based on a preset time interval.

[0082] Specifically, during remote surgery, the video stream acquired by the endoscope is generated in real time as a series of frames. In order to enable the image feature extraction model to perceive changes in the surgical stage in a timely manner without increasing the computational burden, representative single frames are extracted from the video stream at preset time intervals as keyframes, rather than processing every frame in the video stream.

[0083] For example, when the preset time interval is set to 100ms, one frame is extracted from the continuous video stream every 100ms as a key frame, so 10 key frames can be obtained per second for subsequent feature extraction and stage determination.

[0084] The second step involves inputting the keyframes into a lightweight convolutional neural network to extract instrument and tissue features from the keyframes, thereby determining the probability of each surgical stage to which each keyframe belongs, and identifying the surgical stage label with the highest probability as the current surgical stage label.

[0085] Specifically, after extracting keyframes from the video stream, these keyframes are fed into a lightweight convolutional neural network (CNN) for processing. The CNN extracts instrument and tissue features from the keyframes layer by layer through its internal convolutional and pooling layers, such as the contour shape of the instrument's end, the edge texture of the tissue, and the relative positional relationship between the instrument and the tissue. Based on these extracted visual features, the CNN calculates the probability value of each keyframe belonging to a predefined surgical stage at its output layer, obtaining a set of probability distributions representing the degree of matching between the current image and each surgical stage. Finally, the surgical stage label with the highest probability value is determined as the current surgical stage label.

[0086] Optionally, the specific implementation of determining the label for the current surgical stage may include the following steps: (1) The most recent keyframes collected in succession form an observation sequence.

[0087] The observation sequence refers to a set of keyframes composed of multiple recently acquired keyframes arranged in chronological order.

[0088] Specifically, during the process of extracting keyframes from the video stream at preset time intervals, after each new keyframe is extracted, it is arranged together with multiple recently acquired keyframes in the order in which they were extracted, forming an observation sequence for the current stage of judgment.

[0089] (2) Input each frame in the observation sequence into a lightweight convolutional neural network to obtain the posterior probability vector of each surgical stage to which each key frame belongs.

[0090] The posterior probability vector is a vector composed of probability values ​​representing the likelihood that a keyframe belongs to each of the preset surgical stages.

[0091] Specifically, each keyframe in the observation sequence is fed into a lightweight convolutional neural network for processing. For each keyframe in the observation sequence, the lightweight convolutional neural network outputs a posterior probability vector, which records the probability value of the keyframe belonging to each preset surgical stage. Each frame in the observation sequence undergoes the above processing, ultimately resulting in several posterior probability vectors, the same number as the number of keyframes in the observation sequence. Each vector corresponds to an independent result for one keyframe in the observation sequence.

[0092] (3) Accumulate the posterior probability vectors of all keyframes in the observation sequence and normalize them, and determine the surgical stage label with the highest cumulative total probability as the current surgical stage label.

[0093] Specifically, the posterior probability vectors corresponding to each keyframe in the observation sequence are cumulatively summed. That is, for each surgical stage, the probability values ​​of each frame in the observation sequence at that stage are added together to obtain a cumulative probability vector. This cumulative probability vector is then normalized so that the sum of the cumulative probabilities of each stage is normalized to a unit value. After normalization, the cumulative total probabilities corresponding to each surgical stage are compared, and the surgical stage label with the highest cumulative total probability is determined as the current surgical stage label. In this way, misjudgments that might occur in a single keyframe due to momentary occlusion or blurring are compensated for by the judgment results of other frames in the observation sequence, making the final determined current surgical stage label more stable and reliable.

[0094] S240. Based on network status awareness information, surgical scene awareness information, and preset surgical safety constraints, determine the adjustment scheme for image transmission parameters.

[0095] S250. According to the adjustment plan, adjust the image transmission parameters and output the image transmission control signal.

[0096] The technical solution of this application embodiment, when determining surgical scene perception information and network state perception information, uses a temporal feature extraction model and an image feature extraction model to process network state parameters and video stream keyframes in parallel, respectively. These processes are independent of each other and do not block each other, so that the acquisition of network state perception information and surgical scene perception information can proceed in parallel in time, avoiding the additional delay caused by serial processing.

[0097] Example 3: Figure 4 This diagram illustrates an adaptive control method for image transmission in a remote surgical robot, as provided in this embodiment. Based on the foregoing embodiments, this embodiment elaborates on the specific determination method for adjusting image transmission parameters. For detailed implementation methods, please refer to the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here. Figure 4 As shown, the method specifically includes the following steps: S310: Acquire the video stream captured by the endoscope during remote surgery, as well as the network status parameters of the current network link.

[0098] S320 performs parallel sensing processing on the video stream and network status parameters, and outputs surgical scene sensing information and network status sensing information.

[0099] S330. Determine the upper limit constraint of the bit rate based on the low-bit estimate of the bandwidth in the network state awareness information.

[0100] Among them, the upper limit constraint of bit rate refers to the highest limit that the bit rate in the image transmission parameters cannot exceed.

[0101] Specifically, the low bandwidth estimate in network state awareness information represents the minimum guaranteed bandwidth available to the current network link over a future period. This value directly determines the upper limit of the bitrate that the video stream can achieve during transmission. The bitrate upper limit constraint is determined based on this low bandwidth estimate, i.e., the upper limit of the bitrate is set to not exceed a certain percentage of the low bandwidth estimate, thereby reserving sufficient bandwidth margin for control signals and audio transmission.

[0102] S340. Based on the surgical stage labels in the surgical scene perception information, determine the allocation method of resolution weight and frame rate weight in the image quality optimization objective.

[0103] Among them, the image quality optimization target refers to the comprehensive evaluation criteria used to measure the quality of video streams under different parameter combinations.

[0104] The allocation method of resolution weight and frame rate weight refers to the differentiated configuration scheme based on the importance of resolution and frame rate in the image quality optimization goal at the current surgical stage.

[0105] Specifically, the surgical stage labels in the surgical scene perception information represent the specific operational stage of the current surgery. Different surgical stages have different requirements for video stream image quality. For example, the delicate suturing stage requires high resolution clarity, while the tissue separation stage requires smoother visual movements. Based on the operational requirements corresponding to the current surgical stage label, resolution scores and frame rate scores can be assigned different levels of importance in the image quality optimization objectives.

[0106] For example, different surgical stages have different requirements for image quality, so the values ​​of resolution weight and frame rate weight will be dynamically adjusted according to the surgical stage label. When the surgical stage label is suturing and anastomosis stage, since this stage requires clear identification of tissue layers and suture direction, the resolution weight is set to the highest value of 0.8 and the frame rate weight is set to the lowest value of 0.1 to prioritize the detail of the image. When the surgical stage label is separation and dissection stage or lens wiping and movement stage, since this stage requires smooth image tracking of instrument movement, the resolution weight is set to the lowest value of 0.1 and the frame rate weight is set to the highest value of 0.8 to prioritize the smoothness of the image. When the surgical stage label is hemostasis and irrigation stage, the resolution weight and frame rate weight are set to equal intermediate values, i.e., 0.45 each, allowing a certain range of small fluctuations in image quality. When the surgical stage label is dangerous area operation stage, the resolution weight is set to a higher value of 0.7 and the frame rate weight is set to a lower value of 0.2. At this time, high image quality is forcibly maintained and parameter degradation is not allowed to ensure surgical safety.

[0107] Through the above method, the allocation method enables the image quality optimization target to adaptively adjust the weight emphasis according to the actual needs of the current surgical stage, ensuring that the optimization direction of the adjustment scheme matches the needs of the surgical operation.

[0108] S350: Based on preset surgical safety constraints, bit rate upper limit constraints, and allocation methods, select the parameter combination that satisfies the constraints and has the highest image quality score from multiple candidate parameter combinations as the image transmission parameter adjustment scheme.

[0109] Here, a candidate parameter combination refers to a set of alternative parameters composed of various image transmission parameters combined with different values. Optionally, the candidate parameter combination may include at least one image transmission parameter selected from bitrate, resolution, frame rate, buffer size, and forward error correction redundancy rate.

[0110] Bitrate refers to the amount of data generated per unit time during the encoding and compression process of a video stream; resolution refers to the number of pixels in each frame of the video stream in the horizontal and vertical directions; frame rate refers to the number of image frames displayed per unit time in a video stream; buffer size refers to the capacity of the buffer storage area set in the image transmission module to smooth network jitter; and forward error correction redundancy rate refers to the proportion of redundant data added during video stream transmission to cope with data packet loss relative to the original data.

[0111] The forward error correction redundancy rate is positively correlated with the filtered packet loss rate in a stepped manner, with different packet loss rate ranges corresponding to different redundancy rate levels. The forward error correction redundancy rate increases progressively with the filtered packet loss rate, but the relationship is not linear. Instead, it is divided into different redundancy rate levels according to preset packet loss rate ranges, with each range corresponding to a fixed redundancy rate value or a smaller range of values. For example, when the filtered packet loss rate is below 0.5%, the forward error correction redundancy rate is set to the lowest level, 2%; when the filtered packet loss rate is between 0.5% and 2%, the forward error correction redundancy rate is increased to the middle level, between 5% and 10%; when the filtered packet loss rate is between 2% and 5%, the forward error correction redundancy rate is further increased to between 10% and 15%; when the filtered packet loss rate is not lower than 5%, the forward error correction redundancy rate is set to the highest level, 15%, simultaneously triggering a downgrade operation of the bitrate, resolution, or frame rate. Through the above-mentioned step-by-step correspondence, the forward error correction redundancy rate can adaptively adjust the redundancy protection strength according to the network packet loss situation.

[0112] Specifically, after determining the bitrate cap constraint and allocation method, the final target configuration needs to be selected from all possible combinations of image transmission parameters. Pre-defined surgical safety constraints specify hard boundaries that must be met for end-to-end latency, image quality baseline, buffer and jitter matching, and forward error correction redundancy. The bitrate cap constraint further limits the bitrate to the bandwidth guarantee value that the current network link can carry. Together, these constitute the necessary conditions that must be met simultaneously during the parameter combination selection process. Based on this, the weighting relationships determined in the allocation method are applied to each candidate parameter combination, and its overall image quality score is calculated. Among all candidate parameter combinations that simultaneously satisfy the pre-defined surgical safety constraints and the bitrate cap constraint, the parameter combination with the highest overall image quality score is selected as the image transmission parameter adjustment scheme.

[0113] In this embodiment, optionally, the specific implementation of selecting the parameter combination that satisfies the constraints and has the highest image quality score from multiple candidate parameter combinations may include the following steps: The first step is to obtain the preset surgical safety constraints.

[0114] Among them, the preset surgical safety constraints include at least one of the following: end-to-end latency constraints, image quality baseline constraints, buffer size and jitter matching constraints, and forward error correction redundancy rate dynamic adjustment constraints.

[0115] In this embodiment, the end-to-end latency constraint refers to the restriction that the total time taken for the video stream from acquisition and encoding to decoding and display must not exceed a preset safety threshold. For example, the end-to-end latency constraint is: the sum of the encoding latency, the one-way latency after filtering, and the buffer size must not exceed a preset end-to-end latency threshold; Image quality baseline constraints refer to the restrictions that resolution, frame rate, and bit rate must not be lower than their respective preset minimum thresholds. For example, image quality baseline constraints are: resolution not lower than the preset minimum resolution threshold, frame rate not lower than the preset minimum frame rate threshold, and bit rate not lower than the preset minimum bit rate threshold.

[0116] Buffer size matching constraints refer to the restrictions that require the buffer size to match the network latency jitter and be limited to a preset range. For example, a buffer size matching constraint could be: the buffer size is positively correlated with the filtered latency jitter, and the buffer size is limited to a preset minimum and maximum buffer value.

[0117] The forward error correction redundancy rate dynamic adjustment constraint refers to the restriction condition that dynamically adjusts the forward error correction redundancy rate level according to the changes in the network packet loss rate.

[0118] The second step is to calculate the overall image quality score for each candidate parameter combination according to the allocation method, while simultaneously meeting the surgical safety constraints and the upper limit of the bit rate. The candidate parameter combination with the highest overall image quality score is then determined as the optimal adjustment scheme for the image transmission parameters.

[0119] Among them, the overall image quality score refers to the comprehensive score value used to quantitatively evaluate the overall image quality of the candidate parameter combination.

[0120] Specifically, for all possible candidate parameter combinations, the process first filters them one by one according to the hard constraints and bitrate limit constraints in the preset surgical safety constraints, eliminating any combinations that do not meet any of these constraints, and retaining only candidate combinations that simultaneously meet all constraints. Based on this, the overall image quality score of each of these retained combinations is calculated according to the weighted relationships determined in the allocation method. Finally, the candidate parameter combination with the highest overall image quality score is selected as the optimal image transmission parameter adjustment scheme. Through this process, the final determined adjustment scheme ensures both strict satisfaction of all safety and bandwidth constraints and relatively optimal image quality within feasible limits.

[0121] More specifically, under the premise of simultaneously satisfying surgical safety constraints and bitrate upper limit constraints, the specific implementation steps for calculating the comprehensive image quality score of each candidate parameter combination according to the allocation method, and determining the candidate parameter combination with the highest comprehensive image quality score as the optimal image transmission parameter adjustment scheme include: (1) Based on surgical safety constraints and bit rate upper limit constraints, the candidate parameter combinations are pruned to exclude candidate parameter combinations that do not meet any of the constraints.

[0122] Specifically, among all possible combinations of candidate parameters, surgical safety constraints and bitrate cap constraints constitute hard boundaries that must be satisfied simultaneously. To efficiently select feasible solutions from a large number of candidate combinations, each hard constraint in the surgical safety constraints and the bitrate cap constraint are first applied to all candidate parameter combinations one by one. Combinations that cannot satisfy any one of these constraints are eliminated, and only candidate combinations that simultaneously satisfy all constraints are retained.

[0123] Optionally, pruning the candidate parameter combinations to exclude those that do not meet any of the constraints can be implemented in the following ways: based on the image quality baseline constraint in the surgical safety constraints, exclude candidate parameter combinations whose resolution, frame rate, or bitrate is lower than their respective minimum thresholds; based on the end-to-end latency constraint in the surgical safety constraints, exclude candidate parameter combinations whose encoding latency, filtered one-way latency, and buffer size sum exceed the end-to-end latency threshold; based on the buffer size and jitter matching constraint in the surgical safety constraints, exclude candidate parameter combinations whose buffer size does not match the filtered latency jitter or whose buffer size exceeds a preset range; and based on the bitrate upper limit constraint, exclude candidate parameter combinations whose bitrate exceeds the upper limit value.

[0124] In this embodiment, during the pruning process for candidate parameter combinations, various constraints are used to screen each candidate combination from different dimensions. The image quality baseline constraint checks whether the resolution, frame rate, and bitrate of each candidate combination are all below their respective preset minimum thresholds, excluding combinations where any indicator is below the threshold. The end-to-end latency constraint calculates whether the sum of the encoding latency, the filtered one-way latency, and the buffer size exceeds a preset end-to-end latency threshold, excluding combinations that exceed this threshold. The buffer size and jitter matching constraint checks whether the buffer size matches the filtered latency jitter and whether it is between a preset minimum and maximum value, excluding combinations that do not match or exceed this range. The bitrate upper limit constraint checks whether the bitrate exceeds an upper limit determined based on a low-bit bandwidth estimate, excluding combinations that exceed this limit. After these layers of screening by various constraints, the remaining candidate parameter combinations are all feasible solutions that simultaneously satisfy all hard constraints.

[0125] Through the above pruning process, the search space for candidate parameter combinations is significantly reduced, so that subsequent calculation and comparison of the overall image quality score only need to be performed on the remaining feasible combinations, which greatly reduces the amount of computation.

[0126] (2) Based on the remaining candidate parameter combinations, determine the resolution score, frame rate score and image stability score for each candidate parameter combination.

[0127] The resolution score is a quantified score assigned based on the resolution level in the candidate parameter combination. For example, the resolution score is determined by the resolution level in the candidate parameter combination; a higher resolution results in a higher resolution score. The frame rate score is a quantified score assigned based on the frame rate level in the candidate parameter combination. For example, the frame rate score is determined by the frame rate level in the candidate parameter combination; a higher frame rate results in a higher frame rate score. The image stability score is a quantified score assigned based on the degree of frame rate fluctuation and stuttering during video stream transmission, reflecting the smoothness of the video playback.

[0128] Specifically, after pruning, the remaining candidate parameter combinations are all feasible solutions that simultaneously satisfy all constraints. For these remaining candidate parameter combinations, their performance across various dimensions of image quality needs to be further evaluated to rank them. Specifically, for each candidate parameter combination, a resolution score is determined based on its included resolution level, a frame rate score is determined based on its included frame rate level, and a picture stability score is determined based on the degree of frame rate fluctuation and stuttering during video stream transmission. This results in each candidate parameter combination corresponding to a three-dimensional scoring vector consisting of resolution score, frame rate score, and picture stability score.

[0129] Optionally, the image stability score is determined as follows: based on the frame rate variance within a preset time window and the number of consecutive stuttering frames below the target frame rate, the image stability score is determined according to a preset quantization grading rule.

[0130] The preset time window refers to a predetermined continuous time period used to statistically analyze frame rate fluctuations and stuttering. Frame rate variance refers to the degree of dispersion of the actual frame rate of each frame relative to the target frame rate within this preset time window.

[0131] The number of stuttering frames that are continuously lower than the target frame rate refers to the number of frames whose actual frame rate is continuously lower than the target frame rate within the preset time window.

[0132] The quantization grading rules refer to pre-defined grading criteria used to map different combinations of frame rate variance and the number of consecutive stuttering frames below the target frame rate to different levels of image stability scores. Specifically, the quantization grading rules include: when the frame rate variance does not exceed a first preset threshold and there are no consecutive second preset number of frames or more below the target frame rate, it is determined to be a first preset score; when the frame rate variance is greater than the first preset threshold but not greater than the third preset threshold, or there are consecutive fourth to fifth preset number of frames below the target frame rate and the frame rate of a single frame is not less than a sixth preset percentage of the target frame rate, it is determined to be a second preset score; when the frame rate variance is greater than the third preset threshold, or there are consecutive seventh preset number of frames or more below the target frame rate, or the frame rate of a single frame is less than an eighth preset percentage of the target frame rate, it is determined to be a third preset score.

[0133] Specifically, when evaluating screen stability, two quantitative indicators are required: frame rate variance and the number of consecutive stuttering frames below the target frame rate, within a preset time window. Frame rate variance reflects the degree of fluctuation of the actual frame rate relative to the target frame rate within this time window, while the number of consecutive stuttering frames below the target frame rate reflects the severity of screen stuttering. Based on the specific values ​​of these two indicators, the current screen stability level is determined according to a pre-set quantitative grading rule, and the preset score corresponding to this level is determined as the screen stability score.

[0134] For example, when determining the screen stability score, the screen stability is divided into three levels based on two indicators: the frame rate variance obtained statistically within a preset time window and the number of stuttering frames that are continuously lower than the target frame rate, according to a pre-set quantization and grading rule. When the frame rate variance does not exceed the first preset threshold (value 5) and there are no consecutive second preset number of frames (two or more consecutive frames below the target frame rate), the image transmission is smooth and without stuttering, and the first preset score (100 points) is determined. When the frame rate variance is greater than the first preset threshold (5) but does not exceed the third preset threshold (value 15), or there are consecutive fourth to fifth preset number of frames (two to four consecutive frames below the target frame rate) and the single frame rate is not lower than the sixth preset percentage (80%) of the target frame rate, the image has slight stuttering but is still acceptable, and the second preset score (80 points) is determined. When the frame rate variance is greater than the third preset threshold (value 15), or there are consecutive seventh preset number of frames (five or more consecutive frames below the target frame rate), or the single frame rate is lower than the eighth preset percentage (60%) of the target frame rate, the image has severe stuttering, and the third preset score (40 points) is determined.

[0135] For example, within a preset time window (the past 3 seconds), if the statistically obtained frame rate variance is 3 and there are no two or more consecutive frames below the target frame rate, it is judged as no stuttering, and the picture stability score is 100 points; if the frame rate variance is 10, or there are three consecutive frames below the target frame rate and the frame rate of a single frame is not lower than 80% of the target frame rate, it is judged as slight stuttering, and the picture stability score is 80 points; if the frame rate variance is 20, or there are five consecutive frames below the target frame rate, or the frame rate of a single frame is lower than 60% of the target frame rate, it is judged as severe stuttering, and the picture stability score is 40 points. The above-mentioned frame rate variance and the number of stuttering frames consecutively below the target frame rate are statistically analyzed in real time by the remote video decoding module and periodically transmitted back through the reverse control channel to update the picture stability score.

[0136] (3) Based on the resolution weight and frame rate weight corresponding to the surgical stage label in the surgical scene perception information, the resolution score, frame rate score and image stability score are weighted and summed to obtain the comprehensive image quality score of each candidate parameter combination.

[0137] Specifically, based on the surgical stage label in the surgical scene perception information, the resolution weight and frame rate weight corresponding to that label are determined. Meanwhile, the weight corresponding to the image stability score remains fixed. The resolution score is multiplied by its resolution weight, the frame rate score by its frame rate weight, and the image stability score by its fixed weight. These three products are then summed to obtain the overall image quality score for that candidate parameter combination. Through this weighted summation method, the emphasis on image quality requirements of the surgical stage label is passed to the calculation of the overall image quality score, enabling the overall score to reflect the differentiated requirements of different image quality dimensions at the current surgical stage.

[0138] It should be noted that the sum of the fixed weights corresponding to resolution weight, frame rate weight, and image stability is 1.

[0139] (4) Compare the overall image quality scores of each candidate parameter combination, and select the candidate parameter combination with the highest score as the optimal image transmission parameter adjustment scheme.

[0140] Specifically, after calculating the overall image quality score for each candidate parameter combination, each retained candidate parameter combination corresponds to an overall score value. This score quantifies the degree to which the combination is adapted to the needs of the current surgical stage while satisfying all constraints. By comparing the overall image quality scores of each candidate parameter combination, the candidate parameter combination with the highest score is selected and output as the optimal image transmission parameter adjustment scheme.

[0141] S360. According to the adjustment plan, adjust the image transmission parameters and output the image transmission control signal.

[0142] The technical solution of this embodiment, when determining the adjustment scheme of image transmission parameters, transforms the low-bit bandwidth estimate in the network state perception information into a bitrate upper limit constraint to ensure that the adjustment scheme never exceeds the lower limit of the actual bandwidth that the current network link can carry. Simultaneously, the surgical stage label in the surgical scene perception information is transformed into a resolution weight and frame rate weight allocation method, allowing the optimization objective to adaptively adjust its emphasis according to the different image quality requirements of different surgical stages. Based on this, candidate parameter combinations are pruned and screened using preset surgical safety constraints and bitrate upper limit constraints as hard boundaries. The weighted summation of the overall image quality score is then used as the selection criterion. This ensures that all safety constraints are strictly met while maximizing the matching degree with the current surgical stage requirements within a feasible range, achieving optimal image quality under the premise of safety and compliance.

[0143] Example 4: Figure 5This diagram illustrates an adaptive control method for image transmission in a remote surgical robot, as provided in this embodiment. Based on the foregoing embodiments, the implementation of adjusting image transmission parameters according to an adjustment scheme can be further described in detail. Specific implementation methods can be found in the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0144] like Figure 5 As shown, the method specifically includes the following steps: S410: Acquire the video stream captured by the endoscope during remote surgery, as well as the network status parameters of the current network link.

[0145] S420 performs parallel sensing processing on the video stream and network status parameters, and outputs surgical scene sensing information and network status sensing information.

[0146] S430: Based on network status awareness information, surgical scene awareness information, and preset surgical safety constraints, determine the adjustment scheme for image transmission parameters.

[0147] S440: Based on the resolution and frame rate in the adjustment scheme, the parameters are adjusted according to the preset progressive adjustment strategy; the bitrate adjustment is performed linearly and finely by an amount not exceeding the preset percentage of the current bitrate.

[0148] The progressive adjustment strategy refers to a parameter adjustment method in which the resolution, frame rate and bit rate in image transmission parameters are gradually approached to the target value according to their respective preset step size and adjustment interval, rather than jumping directly to the target value all at once.

[0149] Specifically, after determining the adjustment plan, the target values ​​for resolution, frame rate, and bitrate need to be implemented in the actual parameter switching of the image transmission module. For resolution and frame rate, a preset progressive adjustment strategy is adopted, changing only one level each time to avoid sudden changes in resolution or frame rate that could cause visual shock to the image. For bitrate, linear fine-tuning is performed at a rate not exceeding a preset percentage of the current bitrate, allowing the bitrate to gradually approach the target value in the adjustment plan in small steps. Through the above parameter-based, differentiated, and progressive adjustment method, resolution, frame rate, and bitrate each complete parameter switching gradually according to their own characteristics, ensuring a smooth transition throughout the adjustment process without any sudden changes in the image.

[0150] For example, resolution adjustments are made in increments of no more than one level at a time, such as gradually decreasing from 4K to 1080p or gradually increasing from 1080p to 4K; frame rate adjustments are also made in increments of no more than one level at a time, such as gradually decreasing from 60fps to 30fps or gradually increasing from 30fps to 60fps; bitrate adjustments are made linearly in increments of no more than 10% of the current bitrate.

[0151] In this embodiment, optionally, the specific implementation of parameter adjustment according to the preset progressive adjustment strategy may include: for resolution upgrade operation, the resolution is increased by one level only when the low bandwidth estimate in the network state awareness information meets the upgrade condition for a continuous first preset duration; for resolution downgrade operation, the resolution is decreased by one level only when the low bandwidth estimate in the network state awareness information meets the downgrade condition for a continuous second preset duration; frame rate adjustment is performed according to the same hysteresis determination logic as resolution adjustment.

[0152] The first preset duration is longer than the second preset duration, meaning the observation window for upgrade conditions is longer than that for downgrade conditions, making parameter upgrades more cautious than downgrades.

[0153] Specifically, when adjusting parameters according to a preset progressive adjustment strategy, resolution and frame rate adjustments employ hysteresis-based decision logic to avoid frequent parameter fluctuations. For resolution upgrades, whether the upgrade conditions are met depends on whether the low-order bandwidth estimate in the network state awareness information reaches the threshold allowing for a resolution increase. However, this condition needs to be met continuously for a first preset duration before triggering the actual resolution upgrade action, i.e., increasing the resolution by one level. For resolution downgrades, whether the downgrade conditions are met depends on whether the low-order bandwidth estimate in the network state awareness information has decreased to the threshold requiring a resolution reduction. However, this condition only needs to be met continuously for a second preset duration to trigger the actual resolution downgrade action, i.e., decreasing the resolution by one level. Frame rate adjustments are executed using the same hysteresis-based decision logic as resolution adjustments; that is, frame rate upgrades require meeting the upgrade condition for a continuous first preset duration, while frame rate downgrades only require meeting the downgrade condition for a continuous second preset duration.

[0154] By using the above method, the upgrade and downgrade use asymmetrical judgment durations, which allows the resolution and frame rate to be increased more cautiously when the network condition improves to ensure stability, and to be reduced relatively quickly when the network condition deteriorates to ensure the continuity of video stream transmission. This avoids frequent fluctuations in the picture caused by repeated switching of parameters near the network condition threshold.

[0155] S450 outputs an image transmission control signal containing adjusted image transmission parameters.

[0156] Specifically, after the parameter adjustment operation is completed, the resolution, frame rate, and bitrate have gradually transitioned to near the target values ​​specified in the adjustment scheme according to the progressive adjustment strategy. At this point, the specific values ​​of the adjusted image transmission parameters, including the final determined resolution level, frame rate level, and bitrate values, are encapsulated in the form of an image transmission control signal. This signal is transmitted to the image transmission module as the final output of the entire adaptive control process, driving the image transmission module to perform actual encoding and transmission configuration according to the parameter values ​​carried in the signal.

[0157] The technical solution of this embodiment, when adjusting image transmission parameters according to the adjustment scheme, combines incremental adjustments to resolution and frame rate with linear fine-tuning of bitrate. This allows the image transmission parameters to gradually approach the target values ​​in the adjustment scheme during the actual switching process, effectively avoiding the visual impact and stuttering caused by abrupt parameter changes in the video stream. Simultaneously, the linear fine-tuning of the bitrate ensures a smooth change in the amount of encoded data, further reducing drastic fluctuations in network transmission pressure. Through this gradual parameter adjustment process, the actual parameter switching driven by the final output image transmission control signal is smooth and without abrupt changes, achieving both the target configuration of the adjustment scheme and ensuring the continuity and viewing comfort of the video stream during the adjustment transition.

[0158] During the parameter adjustment process, the resolution, frame rate, and bitrate smoothly approach the target values ​​specified in the adjustment scheme according to a progressive adjustment strategy. However, in actual deployment scenarios with multiple network transmission links, optimizing the encoding parameters alone is insufficient to fully guarantee the stability of video stream transmission; it is also necessary to consider how to rationally allocate different types of data carrying the video stream to different links. Therefore, when multiple available network links exist, this method further introduces a multi-link collaborative allocation mechanism. Specifically, it may include the following steps: The first step is to obtain the bandwidth and latency status of each available network link.

[0159] In this context, an available network link refers to a network connection channel that is currently connected and can be used for video streaming data transmission. Latency refers to the time it takes for a data packet to travel unidirectionally from the sender to the receiver via this link.

[0160] Specifically, when multiple network transmission links exist, it is necessary to determine the current transmission capacity of each available network link as the basis for data allocation. For each currently available network link, its bandwidth status and latency status are obtained. The bandwidth status reflects the amount of data transmission that the link can carry per unit time, and the latency status reflects the time taken for a data packet to be transmitted one-way from the sender to the receiver via the link.

[0161] The second step is to allocate the video stream data in the adjusted image transmission parameters to links with sufficient bandwidth and key frame data to links with sufficient latency, based on the bandwidth and latency status of each link.

[0162] Keyframe data refers to the data content corresponding to the frames in the video stream that carry complete picture information.

[0163] Specifically, after obtaining the bandwidth and latency status of each available network link, it is necessary to rationally allocate transmission conditions based on the differentiated requirements of different types of data. For video stream data corresponding to the adjusted image transmission parameters, since the amount of data generated during transmission is large, it should be allocated to links whose bandwidth status meets the transmission requirements to ensure that large-volume data can obtain sufficient bandwidth resources without causing link congestion.

[0164] For keyframe data, because it carries complete image reconstruction information in video decoding, the real-time requirements for transmission are higher. It is allocated to links with acceptable latency to ensure that keyframes arrive at the receiving end with low latency. Through this differentiated allocation method, video stream data and keyframe data are routed to their respective suitable links for transmission.

[0165] The third step is to output an image transmission control signal that includes the adjusted image transmission parameters and the multi-link allocation results.

[0166] Among them, the multi-link allocation result refers to the data distribution strategy determined after allocating different types of data to be transmitted in the video stream according to the bandwidth and latency status of each available network link.

[0167] Specifically, after completing the multi-link data allocation, the adjusted image transmission parameters and the data distribution strategy determined by the multi-link allocation result together constitute all the instruction information required to drive the image transmission module to perform a complete adjustment. These two parts of information are encapsulated together into a final image transmission control signal and output. This signal contains not only the final values ​​of encoding-side parameters such as resolution, frame rate, and bit rate, but also allocation instructions on which link the video stream data and keyframe data should use for transmission. This allows the image transmission module to simultaneously perform adjustments at both the encoding parameter switching and multi-link data distribution levels upon receiving this signal.

[0168] After the parameters are adjusted according to the adjustment plan and the image transmission control signal is output, the video stream will be encoded and transmitted according to the adjusted parameters. However, dynamic changes in network status during actual transmission may still cause abnormalities such as decreased clarity in the video stream. Therefore, it is necessary to continuously monitor the feedback of the actual transmission effect after the parameter adjustment is completed. Specific implementation methods may include: real-time monitoring of the image clarity status; if the clarity status is lower than a preset clarity threshold, increasing the forward error correction redundancy rate or bitrate; if any of the resolution, frame rate, or bitrate corresponding to the clarity status is lower than a preset image quality baseline threshold, triggering an alarm signal. This can be understood as follows: when the image clarity falls below a preset clarity threshold, it indicates a perceptible degradation in image clarity. In this case, the forward error correction redundancy rate is proactively increased to enhance packet loss resistance, or the bitrate is appropriately increased to improve the encoding accuracy of image details, thereby prioritizing the restoration of image stability. Simultaneously, if any of the resolution, frame rate, or bitrate corresponding to the clarity status falls below a preset minimum image quality threshold, it indicates that the safety margin for surgical procedures has been reached. At this point, an alarm signal is triggered to alert the remote surgeon. Through this closed-loop monitoring and processing, the deviation between the actual transmission effect and the expected result can be continuously sensed after the adjustment plan is implemented, and timely intervention can be initiated. When the image quality reaches the minimum threshold, an alarm is triggered to alert the surgeon, further ensuring the safety and reliability of the video stream during remote surgery.

[0169] After the parameters are adjusted and the image transmission control signal is output according to the adjustment plan, the video stream will be encoded and transmitted according to the adjusted parameters. However, dynamic changes in network status during actual transmission may still cause the end-to-end transmission delay to exceed the preset safety range, thereby affecting the operational safety of remote surgery. Therefore, it is necessary to continuously monitor the actual end-to-end transmission delay after the parameter adjustment is completed, and to perform closed-loop control in conjunction with the end-to-end delay constraints set in the adjustment plan. Specifically, the actual end-to-end transmission delay is monitored in real time based on the end-to-end delay constraints in the adjustment plan and the actual transmission status of the current network link. When the actual end-to-end transmission delay exceeds the preset safety delay threshold, the delay is first reduced by gradually decreasing the buffer size. If the buffer size has been reduced to the preset minimum value and the delay still does not meet the standard, the frame rate and resolution are further reduced sequentially until the actual end-to-end transmission delay meets the safety delay threshold again. Through the above closed-loop monitoring and hierarchical handling mechanism, when the actual transmission delay exceeds the standard, intervention can be prioritized by adjusting the buffer, which has the least impact on image quality. Only when the buffer adjustment capacity is exhausted will the frame rate and resolution be sacrificed step by step, thereby maintaining a high level of image quality as much as possible while ensuring delay safety.

[0170] Example 5: Figure 6This diagram illustrates an adaptive control method for image transmission in a remote surgical robot, as provided in this embodiment. Building upon the aforementioned embodiments, it can further enhance the system by continuously monitoring the consistency between the actual delay and the predicted state. When the model's perception fails, it automatically reverts to a deterministic rule-based mode, ensuring safe operation even when the model is unreliable. Specific implementation details can be found in the technical solution of this embodiment. Technical terms identical or corresponding to those in the above embodiments will not be repeated here.

[0171] like Figure 6 As shown, the method specifically includes the following steps: S510: Acquire the video stream captured by the endoscope during remote surgery, as well as the network status parameters of the current network link.

[0172] S520 performs parallel sensing processing on the video stream and network status parameters, and outputs surgical scene sensing information and network status sensing information.

[0173] S530 determines the adjustment scheme for image transmission parameters based on network status awareness information, surgical scene awareness information, and preset surgical safety constraints.

[0174] S540. According to the adjustment plan, adjust the image transmission parameters and output the image transmission control signal.

[0175] S550. When the deviation between the actual end-to-end transmission delay and the predicted state in the network state perception information exceeds a preset deviation threshold within a preset number of consecutive monitoring cycles, the model perception processing is deemed to have failed.

[0176] Among them, the preset deviation threshold refers to the pre-set threshold used to determine whether the deviation between the actual end-to-end transmission delay and the predicted state in the network state awareness information has reached the minimum deviation threshold that requires triggering the backoff mechanism.

[0177] Among them, model perception processing failure refers to a state in which the predicted state output by the model in parallel perception processing and the actual transmitted state are continuously and systematically deviated from each other by a preset deviation threshold due to data drift or environmental changes, thus making the model perception results unreliable.

[0178] Specifically, after outputting the image transmission control signal, the actual end-to-end transmission delay is continuously monitored, while the network state perception information includes a predicted state of the network. The measured value of the actual end-to-end transmission delay in each monitoring period is compared with the predicted state in the network state perception information, and the deviation between the two is calculated. When this deviation exceeds a preset deviation threshold for a consecutive preset number of monitoring periods, it indicates that the predicted state in the network state perception information has experienced a continuous and unacceptable systematic deviation from the actual transmission state, at which point the model perception processing is deemed to have failed.

[0179] S560, in response to the decision, disables the processing mode in parallel sensing processing and switches to a pure rule mode based on filtering and linear prediction to process network state parameters.

[0180] Specifically, after determining that the model-based perception processing has failed, the original parallel perception processing method can no longer provide reliable perception results, so it is necessary to immediately stop the operation of this processing method. At the same time, the processing method for network state parameters is switched to a pure rule-based mode based on filtering and linear prediction. That is, it no longer relies on the model for perception, but instead uses deterministic filtering and linear prediction algorithms to process the network state parameters, so as to ensure that basic transmission control capabilities can still be maintained even when the network state perception information is unreliable.

[0181] In this embodiment, after the output image transmission control signal, changes in the actual network environment may cause a deviation between the network state perception information output during the parallel perception processing stage and the actual transmission state. Therefore, it is necessary to continuously monitor the consistency between the actual end-to-end transmission delay and the predicted state in the network state perception information. When the deviation between the two exceeds a preset deviation threshold within a preset number of consecutive monitoring periods, it indicates that the model perception processing has failed. At this time, the processing mode in the parallel perception processing is disabled, and the system switches to a pure rule mode based on filtering and linear prediction to process the network state parameters. This allows for timely switching to a deterministic rule mode when the model fails due to data drift, avoiding inappropriate adjustment decisions caused by erroneous perception results and ensuring the safe operation of the system in the event of model failure.

[0182] Example 6: This example is applicable to routine laparoscopic remote surgery scenarios where the latency requirements are moderate. It employs dual-link transmission of 5G and wired connections. Specific implementation details are as follows: Regarding preset parameters, the end-to-end latency threshold is set to 150 milliseconds. The minimum image quality is a resolution of at least 720p, a frame rate of at least 30 frames per second, and a bitrate of at least 4 megabits per second. The encoder adopts the H.265 standard with an encoding latency of 15 milliseconds. The acquisition cycle is fixed at 200 milliseconds to adapt to the timing alignment requirements of the temporal convolutional network model. The lightweight model of the intelligent perception layer is generated into a static library suitable for embedded platform operation through a conversion toolchain and is embedded in the firmware of the embedded video enhancement board in the main control panel of the surgical robot. This board acquires the original video frames of the endoscope through a high-speed data channel and reads network parameters through an Ethernet interface. In terms of model deployment, the temporal convolutional network has 450,000 parameters, the convolutional neural network has 750,000 parameters, and the multilayer perceptron has 280,000 parameters, all deployed in the embedded processor.

[0183] The execution process is as follows: First, a three-layer hybrid control architecture is constructed, with each layer modularly integrated into the existing laparoscopic remote surgical robot system. The intelligent perception layer collects network parameters, and after Kalman filtering, obtains the filtered one-way delay, filtered delay jitter, and filtered packet loss rate; the temporal convolutional network model takes data from the past 10 acquisition cycles as input and predicts the low-order and high-order bandwidth estimates; the convolutional neural network identifies the current surgical stage as the separation stage; the multilayer perceptron outputs a risk probability of 0.3, and does not trigger an early warning. The constraint optimization layer receives the perception parameters and determines the resolution weight as 0.1, the frame rate weight as 0.8, and the image stability weight as 0.1 based on the separation stage; after hard constraint calculation, the optimal parameter combination is solved, and the frame rate variance and stutter count are transmitted back to the constraint optimization layer every 500 milliseconds via real-time transmission control protocol messages to update the stability score. The smooth execution layer detects that the low bandwidth estimate meets the upgrade requirements for 3 consecutive seconds, and gradually adjusts parameters, with the bitrate adjusted by 2 megabits per second each time, and the resolution gradually increased from 720p to 1080p with a transition time of 500 milliseconds. In multi-link allocation, the video stream uses the 5G link, while key frames and control signals use the wired link. No preventative bitrate reduction is performed without warning signals. Real-time monitoring shows that the end-to-end total latency is 90 milliseconds, meeting the latency threshold requirement, and the image quality is stable. When network jitter suddenly increases, the buffer size is dynamically adjusted, and the end-to-end total latency still meets the constraint. The intelligent prediction is found to be in line with the actual state, and the intelligent driving mode is maintained.

[0184] The implementation results are as follows: In conventional laparoscopic remote surgery scenarios, the three-layer hybrid control architecture operates stably, with end-to-end latency consistently kept within 150 milliseconds, image quality maintained at 1080p resolution and 60 frames per second, and dynamic cache adjustment effectively absorbing jitter when the network fluctuates, without any image stuttering or sudden changes, and the intelligent perception layer accurately identifies without requiring manual intervention from the doctor.

[0185] Example 7: This example is applicable to remote surgical scenarios requiring precise suturing. The surgery demands extremely high resolution and low latency, and utilizes a 5G private network for transmission. Specific implementation details are as follows: Regarding preset parameters, the end-to-end latency threshold is set to 100 milliseconds, the minimum image quality is a resolution of at least 720p, a frame rate of at least 30 frames per second, and a bitrate of at least 4 megabits per second. The encoder uses the H.265 standard with an encoding latency of 12 milliseconds, the acquisition period is fixed at 200 milliseconds, and the preset number of temporal convolutional network models is adjusted to 15 to cover a 3-second time window. The intelligent perception layer model is embedded in the video enhancement board and acquires video frames through a high-speed data channel. The model deployment is consistent with Example 1, and the convolutional neural network model is fine-tuned using neurosurgical data.

[0186] The execution process is as follows: First, a three-layer hybrid control architecture is constructed. The intelligent perception layer collects network parameters, and after Kalman filtering, obtains the filtered one-way delay, filtered delay jitter, and filtered packet loss rate; the temporal convolutional network model takes data from the past 15 acquisition cycles as input and predicts the low-order and high-order bandwidth estimates; the convolutional neural network identifies the current surgical stage as the vascular suturing stage; the multilayer perceptron outputs a risk probability of 0.2, and no warning is triggered. The constraint optimization layer receives perception parameters and determines the resolution weight as 0.8, the frame rate weight as 0.1, and the image stability weight as 0.1 based on the suturing stage; after hard constraint calculation, the optimal parameter combination is solved, determining the bitrate as 28.35 megabits per second, the resolution as 4K, the frame rate as 60 frames per second, the buffer size as 24 milliseconds, and the forward error correction redundancy rate as 2%. The smooth execution layer confirms that the parameters meet the upgrade conditions, and gradually switches to 4K resolution and 60 frames per second frame rate, with a transition time of 600 milliseconds, and the bitrate is gradually increased to the target value; there is no warning signal, and the current parameters are maintained. The real-time end-to-end total latency is 56 milliseconds, which meets the latency threshold requirement and the image quality is clear. When the network packet loss rate suddenly increases, the forward error correction redundancy rate is automatically adjusted and the bitrate is finely adjusted to maintain 4K resolution and 60 frames per second. The intelligent prediction is found to be in line with the actual state, and the intelligent driving mode is maintained.

[0187] The results are as follows: In the scenario of remote surgery with fine suturing, the surgical field is clear, and the details of vascular suturing can be clearly distinguished. The 4K resolution and 60 frames per second provide doctors with sufficient visual information; the recognition accuracy of the convolutional neural network model is stable, effectively reducing the frequency of manual intervention by doctors.

[0188] Example 8: This example is applicable to remote surgery scenarios with weak network conditions and unstable network environments. It uses dual-link transmission of 5G and satellite. Specific implementation details are as follows: Regarding preset parameters, the end-to-end latency threshold is set to 150 milliseconds; the minimum image quality is a resolution of at least 720p, a frame rate of at least 30 frames per second, and a bitrate of at least 4 megabits per second; the encoder uses the H.265 standard with an encoding latency of 18 milliseconds; the acquisition period is fixed at 200 milliseconds; the preset number of elements in the temporal convolutional network model is adjusted to 6 to cover a 1.2-second time window; and a low-pass filter is used as the filtering algorithm. The intelligent perception layer model is deployed on an embedded processor, and the multilayer perceptron model focuses on optimizing the risk prediction accuracy in weak network environments.

[0189] The execution process is as follows: First, a three-layer hybrid control architecture is constructed, and a low-pass filter is used in the filtering algorithm to improve the stability of parameters in weak networks. The intelligent perception layer collects network parameters, and after low-pass filtering, the filtered one-way delay, the filtered delay jitter, and the filtered packet loss rate are obtained. The temporal convolutional network model takes data from the past 6 acquisition cycles as input and predicts the low-order and high-order bandwidth estimates. The convolutional neural network identifies the current surgical stage as the hemostasis stage. However, due to the possibility of slight obstruction of the endoscopic field of view in a weak network environment, the convolutional neural network model has a local misjudgment rate of about 5% to 8% in distinguishing between the hemostasis and irrigation stage and the separation and dissection stage. The system automatically downgrades the weight configuration to a conservative medium-balance mode and mainly relies on the doctor's explicit stage specification input through the touch panel. The multilayer perceptron outputs a risk probability of 0.86, triggering an early warning preparation signal to be sent to the smooth execution layer. The constraint optimization layer receives sensing parameters and determines the weights for resolution (0.45), frame rate (0.45), and image stability (0.1) based on the hemostasis stage. After hard constraint calculation, the optimal parameter combination is solved. Due to excessive latency, the buffer size needs to be adjusted. The final determined parameters are: bitrate of 5.0 Mbps, resolution of 720p, frame rate of 30 fps, buffer size of 62 ms, and forward error correction redundancy of 13%. Frame rate variance and stutter count are transmitted back every 500 ms, at which point the stability score is 80. The smoothing execution layer receives a warning signal and preemptively reduces the bitrate by 8% as a preventative buffer. In multi-link allocation, the 5G link is only used for transmitting key frames and control signals due to large fluctuations in weak networks, while the satellite link is used for transmitting the main video stream. Parameter adjustments are made gradually, with the bitrate adjusted by 0.2 Mbps each time, and the transition time is 500 ms. Real-time monitoring showed that the end-to-end total latency was 150 milliseconds, which just met the latency threshold constraint. During operation, a sudden network blockage occurred, causing a sharp increase in packet loss. The forward error correction redundancy rate was automatically adjusted to the highest level, and the bitrate was downgraded to the minimum image quality. The cache size was adjusted, but the end-to-end total latency still met the constraint. Continuous monitoring revealed that the actual end-to-end total latency deviated from the predicted value of the intelligent perception layer by 23 milliseconds within two consecutive cycles, exceeding the preset deviation threshold. It was determined that the intelligent model failed due to data drift caused by the sudden network change. The system automatically reverted to the pure rule constraint mode, disabled intelligent prediction, and switched to Kalman filtering and linear prediction logic. The surgical stage was switched to the doctor's manual marking of the hemostasis stage. After 30 seconds, the network stabilized, the deviation dropped to 5 milliseconds, and the system automatically switched back to the intelligent drive mode.

[0190] The implementation results are as follows: In weak network environments, surgical image transmission did not experience severe stuttering or interruption, the end-to-end latency remained stable in the range of 135 to 150 milliseconds, the image quality was maintained above the minimum of 720p resolution and 30 frames per second, the forward error correction redundancy mechanism effectively compensated for network packet loss, and in the case of local misjudgment, the accuracy of stage recognition was ensured through manual intervention by doctors; the intelligent model failure rollback mechanism met the response time requirements, the surgery proceeded normally during the rollback period, and there were no safety risks.

[0191] Example 9: This example is applicable to complex and fluctuating 5G base station handover scenarios. The core pain point is the drastic fluctuations in network bandwidth, latency, and jitter during 5G base station handover. It uses 5G private network transmission and focuses on demonstrating the system's dynamic adaptability to high-frequency fluctuations, the forward-looking nature of intelligent prediction, and the stability of smooth execution. Specific implementation details are as follows: Regarding preset parameters, the end-to-end latency threshold is set to 150 milliseconds. The minimum image quality is a resolution of at least 720p, a frame rate of at least 30 frames per second, and a bitrate of at least 4 megabits per second. The encoder uses the H.265 standard with an encoding latency of 17 milliseconds. The acquisition cycle is fixed at 200 milliseconds. The preset number of temporal convolutional network models is 8 to cover a 1.6-second time window, balancing prediction accuracy and real-time response speed. The filtering algorithm uses Kalman filtering to adapt to high-frequency fluctuation scenarios. The intelligent perception layer model is fine-tuned using base station handover scenario data, focusing on optimizing the short-term fluctuation prediction accuracy of the temporal convolutional network model and the sensitivity of the multilayer perceptron model in predicting fluctuation scenarios. In terms of model deployment, the temporal convolutional network has 460,000 parameters, the convolutional neural network has 760,000 parameters, and the multilayer perceptron has 285,000 parameters, all deployed on embedded processors.

[0192] The execution process is as follows: First, a three-layer hybrid control architecture is constructed. The smooth execution layer strengthens the hysteresis interval mechanism. Parameter upgrades require the low-order bandwidth estimate to meet the requirements for 3 consecutive seconds, and downgrades require 0.8 consecutive seconds to meet the requirements to adapt to fluctuating scenarios. The multi-link aggregation module only uses 5G private network links, focusing on optimizing parameter caching and smooth transition logic during link switching, and reserving 10% bandwidth redundancy for burst traffic compensation during base station switching. The intelligent perception layer collects initial network parameters, and after Kalman filtering, obtains the filtered one-way delay, filtered delay jitter, and filtered packet loss rate. The temporal convolutional network model predicts the low-order bandwidth estimate and the high-order bandwidth estimate. The convolutional neural network identifies the surgical stage as the separation and peeling stage. The multilayer perceptron outputs a risk probability of 0.32, without triggering an early warning. After 12 seconds of operation, a 5G base station handover is triggered, causing a sudden and drastic fluctuation in network parameters. The intelligent perception layer responds quickly, obtaining filtered one-way delay, filtered delay jitter, and filtered packet loss rate after Kalman filtering. The temporal convolutional network model urgently predicts the low-order and high-order bandwidth estimates. The multilayer perceptron calculates the risk probability in real time to be 0.88, triggering an early warning preparation signal that is immediately sent to the smoothing execution layer. Due to network fluctuations, the convolutional neural network model experiences temporary blurring of the endoscopic video frames, temporarily reducing the recognition accuracy to 90.5%. The system automatically increases the weight of manual intervention by the doctor. The constraint optimization layer solves for the optimal parameters in two stages: before and after the handover. Before the handover, based on the separation stage, the resolution weight is determined to be 0.1, the frame rate weight to be 0.8, and the image stability weight to be 0.1. After hard constraint calculation, the optimal parameter combination is solved. After the handover, it is still in the separation stage, and the optimal parameter combination is solved after hard constraint calculation. Before the handover, the smooth execution layer detects that the low bandwidth estimate meets the upgrade requirements for 3 consecutive seconds and gradually adjusts the parameters. After the handover, it receives an early warning signal and reduces the bit rate by 9% in advance as a preventive buffer. The parameter adjustment adopts a gradual approach, reducing the resolution from 1080p to 720p and the frame rate from 60 frames per second to 30 frames per second. At the same time, the link cache compensation mechanism is enabled to cache 3 key images before the base station handover to avoid the interruption of the picture at the moment of handover. In terms of real-time monitoring and dynamic adjustment, the total end-to-end latency during base station handover was 150 milliseconds, which just met the constraints. The forward error correction redundancy mechanism effectively compensated for packet loss, and there was no obvious stuttering in the picture. After the handover, the network parameters gradually recovered, the temporal convolutional network model re-predicted the low-order bandwidth estimate, and the constraint optimization layer re-solved the optimal parameters. The smooth execution layer detected that the low-order bandwidth estimate met the requirements for 3 consecutive seconds, and gradually improved the parameters to restore the picture quality and frame rate to the level before the handover. The status of the intelligent model was monitored throughout the process, and the deviation was 5 milliseconds, which did not exceed the preset deviation threshold. The intelligent model did not fail and maintained the intelligent driving mode. The recognition accuracy of the convolutional neural network model rebounded rapidly after the handover, and the system automatically restored the intelligent recognition weights.

[0193] The implementation results are as follows: In the 5G base station handover scenario, the system successfully coped with the drastic fluctuations in bandwidth, which dropped sharply from 30 megabits per second to 8 megabits per second and then rebounded. During the handover, the end-to-end latency remained stable in the range of 145 to 150 milliseconds, with no obvious sudden changes or interruptions in the picture, and the frequency of stuttering was extremely low, with only one slight stuttering during the handover. The intelligent model's prediction response time for fluctuations met the requirements, and the early warning mechanism effectively avoided the risk of exceeding the latency limit. The forward error correction redundancy mechanism could still ensure image integrity under high packet loss rate, with no screen tearing or frame dropping.

[0194] Example 10: This example is applicable to multi-hospital federated learning fine-tuning and adaptation scenarios. It addresses the differences in network environments and surgical types among different hospitals, highlighting the privacy protection advantages of federated learning and its local fine-tuning adaptation capabilities. It utilizes a three-link transmission system: 5G, wired, and satellite. Specific implementation details are as follows: Regarding preset parameters, the end-to-end latency threshold is set to 150 milliseconds or 100 milliseconds depending on the type of surgery. The minimum image quality is a resolution of no less than 720p, a frame rate of no less than 30 frames per second, and a bitrate of no less than 4 megabits per second. The encoder adopts the H.265 standard with an encoding latency of 16 milliseconds. The acquisition cycle is fixed at 200 milliseconds to adapt to the timing alignment requirements of multi-hospital models. The lightweight model of the intelligent perception layer is generated into a static library suitable for embedded platform operation through a conversion toolchain and is embedded in the firmware of the embedded video enhancement board in the main control console of the surgical robot. This board acquires raw video frames through a high-speed data channel and reads network parameters through an Ethernet interface. In terms of model deployment, the temporal convolutional network has 480,000 parameters, the convolutional neural network has 780,000 parameters, and the multilayer perceptron has 290,000 parameters, all deployed on the embedded processor. The model has undergone four rounds of optimization using a multi-hospital federated learning algorithm, with three tertiary hospitals and two county-level hospitals participating in collaborative training. The newly added county-level hospital used postoperative surgical video data from eight cases for local fine-tuning.

[0195] The execution process is as follows: First, a three-layer hybrid control architecture is constructed, supporting multi-hospital model parameter adaptation. The modular design of each layer allows for rapid portability to different surgical robot platforms. The multi-link aggregation module supports dynamic switching between 5G, wired, and satellite links, automatically adapting link allocation strategies according to the network environment of each hospital. The intelligent perception layer collects network parameters, and after Kalman filtering, obtains the filtered one-way delay, filtered delay jitter, and filtered packet loss rate. The temporal convolutional network model predicts the low-order and high-order bandwidth estimates. After local fine-tuning, the convolutional neural network model achieves an average recognition accuracy of 95.8% to 98.5% on the hospital's test set, identifying the current surgical stage as the suturing stage. The multilayer perceptron outputs a risk probability of 0.25, without triggering an early warning. During the forward inference and gradient calculation process of local fine-tuning, the endoscopic video stream resides only in the embedded board's memory buffer and is released immediately after calculation, without generating any persistent copies in the local storage medium, ensuring patient privacy and security. The constraint optimization layer receives perceived parameters and determines the weights based on the stitching stage: resolution (0.8), frame rate (0.1), and image stability (0.1). After hard constraint calculation, the buffer size needs to be adjusted due to excessive latency. Finally, the optimal parameter combination is solved. The frame rate variance and stutter count are returned every 500 milliseconds, at which point the stability score is 100. The smoothing execution layer detects that the low bandwidth estimate meets the requirements for 3 consecutive seconds and gradually adjusts the parameters. In multi-link allocation, wired links are used to transmit the main video stream, 5G links are used to transmit key frames, and satellite links are used as redundant backups. There are no warning signals, so the current parameters are maintained. Real-time monitoring shows that the end-to-end total latency meets the constraints, and the image quality is clear and stable. The intelligent model's running status is monitored synchronously, and there are no signs of failure. When the hospital network switches to peak hours and bandwidth decreases, the temporal convolutional network model re-predicts the low bandwidth estimate, the constraint optimization layer re-solves for the optimal parameters, and the smoothing execution layer gradually adjusts the bitrate, without any sudden changes in the image. The model receives real-time running data through the feedback module and dynamically optimizes the prediction accuracy.

[0196] The implementation results are as follows: In multi-hospital collaborative scenarios, federated learning effectively achieves multi-center data privacy protection and model performance improvement, and can complete model collaborative optimization without uploading patient privacy data; after local fine-tuning, the model has excellent recognition accuracy on the test set of newly connected hospitals, the system runs stably, the image quality is clear, and doctors can maintain the current parameter configuration by using the image quality lock button according to the needs of fine operation.

[0197] Example 11: Figure 7 This is a schematic diagram of a remote surgical robot image transmission adaptive control device provided in an embodiment of this application. The device includes: The data acquisition module 610 is used to acquire the video stream captured by the endoscope during remote surgery, as well as the network status parameters of the current network link; Parallel perception processing module 620 is used to perform parallel perception processing on the video stream and the network state parameters, and output surgical scene perception information and network state perception information. The constraint optimization decision module 630 is used to determine the adjustment scheme of image transmission parameters based on the network state perception information, the surgical scene perception information, and preset surgical safety constraints. The transmission signal output module 640 is used to adjust the image transmission parameters according to the adjustment scheme and output an image transmission control signal.

[0198] This application provides an adaptive control device for image transmission in a remote surgical robot. The device includes: a data acquisition module for acquiring video streams captured by an endoscope during remote surgery, as well as network status parameters of the current network link; a parallel perception processing module for performing parallel perception processing on the video streams and network status parameters, and outputting surgical scene perception information and network status perception information; a constraint optimization decision module for determining an adjustment scheme for image transmission parameters based on the network status perception information, the surgical scene perception information, and preset surgical safety constraints; and a transmission signal output module for adjusting the image transmission parameters according to the adjustment scheme and outputting an image transmission control signal. By outputting surgical scene perception information and network status perception information through parallel perception processing, and determining the adjustment scheme for image transmission parameters based on the two types of perception information combined with preset surgical safety constraints, the device achieves adaptive decision-making in response to network changes and surgical stages. This solves the problems of low network prediction accuracy and fixed parameter weights in pure rule-driven schemes, and improves the stability and security of remote surgical image transmission in complex network environments.

[0199] Optionally, based on the above-described device, the parallel sensing processing module 620 includes: The network state awareness unit is used to determine the low-order bandwidth estimate of the network bandwidth based on the network state parameters through a time-series feature extraction model, and obtain the network state awareness information. The surgical scene perception unit is used to determine the label of the current surgical stage based on key frames in the video stream through an image feature extraction model, and obtain the surgical scene perception information.

[0200] Based on the above-mentioned device, optionally, a network state sensing unit is used to acquire the network state parameters for a preset number of acquisition cycles to form a historical network state sequence; wherein, the network state parameters include available bandwidth, one-way latency, latency jitter, and packet loss rate; after filtering and preprocessing the historical network state sequence, the time dependency of the network state parameters is extracted through a lightweight temporal convolutional network to determine the low-order bandwidth estimate of the available bandwidth within the future time window.

[0201] Based on the above-mentioned device, optionally, a surgical scene perception unit is used to extract key frames from the video stream based on a preset time interval; input the key frames into a lightweight convolutional neural network to extract instrument features and tissue features in the key frames, so as to determine the probability of each surgical stage to which each key frame belongs, and determine the surgical stage label with the highest probability as the current surgical stage label.

[0202] Based on the above-mentioned device, optionally, the surgical scene perception unit is more specifically used to construct an observation sequence from multiple recently acquired keyframes; input each frame in the observation sequence into the lightweight convolutional neural network to obtain the posterior probability vector of each surgical stage to which each keyframe belongs; accumulate the posterior probability vectors of all keyframes in the observation sequence and perform normalization processing, and determine the surgical stage label with the highest cumulative total probability as the current surgical stage label.

[0203] Optionally, based on the above-described device, the constraint optimization decision module 630 includes: A bit rate constraint determination unit is used to determine the upper limit constraint of bit rate based on the low bit estimate of bandwidth in the network state awareness information; The image quality weight allocation unit is used to determine the allocation method of resolution weight and frame rate weight in the image quality optimization target based on the surgical stage label in the surgical scene perception information. The adjustment scheme determination unit is used to select the parameter combination that satisfies the constraints and has the highest image quality score from multiple candidate parameter combinations based on the preset surgical safety constraints, the bit rate upper limit constraints, and the allocation method, as the adjustment scheme for the image transmission parameters.

[0204] Based on the above-mentioned device, optionally, an adjustment scheme determination unit is used to obtain the preset surgical safety constraints; wherein, the preset surgical safety constraints include at least one of the following: end-to-end latency constraints, image quality baseline constraints, buffer size and jitter matching constraints, and forward error correction redundancy rate dynamic adjustment constraints; under the premise of simultaneously satisfying the surgical safety constraints and the bitrate upper limit constraints, the image quality comprehensive score of each candidate parameter combination is calculated according to the allocation method, and the candidate parameter combination with the highest image quality comprehensive score is determined as the optimal image transmission parameter adjustment scheme.

[0205] Based on the aforementioned device, optionally, the adjustment scheme determination unit is further configured to: prune candidate parameter combinations based on the surgical safety constraints and the bitrate upper limit constraints, eliminating candidate parameter combinations that do not meet any of the constraints; determine the resolution score, frame rate score, and image stability score for each remaining candidate parameter combination; perform a weighted summation of the resolution score, frame rate score, and image stability score based on the resolution weight and frame rate weight corresponding to the surgical stage label in the surgical scene perception information, to obtain the comprehensive image quality score for each candidate parameter combination; compare the comprehensive image quality scores of each candidate parameter combination, and select the candidate parameter combination with the highest score as the optimal image transmission parameter adjustment scheme.

[0206] Based on the aforementioned device, optionally, the adjustment scheme determination unit is further configured to: exclude candidate parameter combinations whose resolution, frame rate, or bitrate is lower than their respective minimum thresholds based on the image quality baseline constraint in the surgical safety constraints; exclude candidate parameter combinations whose encoding delay, filtered one-way delay, and buffer size sum exceed the end-to-end delay threshold based on the end-to-end delay constraint in the surgical safety constraints; exclude candidate parameter combinations whose buffer size does not match the filtered delay jitter or whose buffer size exceeds a preset range based on the buffer size and jitter matching constraint in the surgical safety constraints; and exclude candidate parameter combinations whose bitrate exceeds the upper limit value based on the upper limit constraint.

[0207] Based on the above-mentioned device, optionally, the image stability score is determined as follows: based on the frame rate variance within a preset time window and the number of consecutive stuttering frames below the target frame rate, the image stability score is determined according to a preset quantization grading rule; wherein, the quantization grading rule includes: when the frame rate variance does not exceed a first preset threshold and there are no consecutive second preset number of frames or more below the target frame rate, it is determined as a first preset score; when the frame rate variance is greater than the first preset threshold but not greater than a third preset threshold, or there are consecutive fourth preset number of frames to fifth preset number of frames below the target frame rate and the single frame rate is not less than a sixth preset percentage of the target frame rate, it is determined as a second preset score; when the frame rate variance is greater than a third preset threshold, or there are consecutive seventh preset number of frames or more below the target frame rate, or the single frame rate is less than an eighth preset percentage of the target frame rate, it is determined as a third preset score.

[0208] Optionally, based on the above-described device, the transmission signal output module 640 includes: The parameter progressive adjustment unit is used to perform parameter adjustments based on the resolution and frame rate in the adjustment scheme, respectively, according to a preset progressive adjustment strategy; the bitrate adjustment is performed linearly with a range not exceeding a preset percentage of the current bitrate. The transmission signal output unit is used to output an image transmission control signal containing adjusted image transmission parameters.

[0209] Based on the above-mentioned device, optionally, a parameter progressive adjustment unit is used to, for resolution upgrade operations, increase the resolution by one level only when the low bandwidth estimate in the network status awareness information meets the upgrade condition for a continuous first preset duration; for resolution downgrade operations, decrease the resolution by one level only when the low bandwidth estimate in the network status awareness information meets the downgrade condition for a continuous second preset duration; wherein, the first preset duration is longer than the second preset duration; the frame rate adjustment is performed according to the same hysteresis determination logic as the resolution adjustment.

[0210] Based on the above-mentioned device, optionally, the remote surgical robot image transmission adaptive control device further includes: a risk warning perception module, used to determine the probability of network transmission risk based on the network status parameters and the current cache occupancy rate through a risk prediction model, and obtain the risk warning perception information; Accordingly, the transmission signal output module 640 is further configured to perform a preventative reduction operation on the bit rate in the adjustment scheme when the risk probability in the risk warning perception information exceeds a preset warning threshold.

[0211] Optionally, when multiple network transmission links exist, the transmission signal output module 640 is further configured to acquire the bandwidth and latency status of each available network link; based on the bandwidth and latency status of each link, allocate the video stream data in the adjusted image transmission parameters to the link with sufficient bandwidth, and allocate the key frame data to the link with sufficient latency; and output an image transmission control signal containing the adjusted image transmission parameters and the multi-link allocation results.

[0212] Based on the above-mentioned device, optionally, the remote surgical robot image transmission adaptive control device further includes: a model failure perception module, used to determine that the model perception processing has failed when the deviation between the actual end-to-end transmission delay and the predicted state in the network state perception information exceeds a preset deviation threshold within a consecutive preset number of monitoring cycles; in response to the determination, disabling the processing mode in the parallel perception processing, and switching to a pure rule mode based on filtering and linear prediction to process the network state parameters.

[0213] The remote surgical robot image transmission adaptive control device provided in this application embodiment can execute the remote surgical robot image transmission adaptive control method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of the method execution.

[0214] It is worth noting that the various units and modules included in the above system are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this application.

[0215] Example 12: Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 A block diagram is shown of an exemplary electronic device 70 suitable for implementing embodiments of the present application. Figure 8 The electronic device 70 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this application. Figure 8 As shown, the electronic device 70 is presented in the form of a general-purpose computing device. The components of the electronic device 70 may include, but are not limited to: one or more processors or processing units 701, system memory 702, and bus 703 connecting different system components (including system memory 702 and processing unit 701).

[0216] Bus 703 represents one or more of several bus architectures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Microchannel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus. Electronic device 70 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 70, including volatile and non-volatile media, removable and non-removable media. System memory 702 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 707 and / or cache memory 705. Electronic device 70 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 706 can be used to read and write non-removable, non-volatile magnetic media (… Figure 8 Not shown; usually referred to as a "hard drive"). Although Figure 8Not shown, a disk drive for reading and writing to removable non-volatile disks (e.g., "floppy disks") and an optical disk drive for reading and writing to removable non-volatile optical disks (e.g., CD-ROMs, DVD-ROMs, or other optical media) may be provided. In these cases, each drive may be connected to bus 703 via one or more data media interfaces. Memory 702 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application. A program / utility 708 having a set (at least one) of program modules 707 may be stored, for example, in memory 702. Such program modules 707 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 707 generally perform the functions and / or methods described in the embodiments of this application. Electronic device 70 can also communicate with one or more external devices 709 (e.g., keyboard, pointing device, display 710, etc.), and with one or more devices that enable a user to interact with the electronic device 70, and / or with any device that enables the electronic device 70 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 711. Furthermore, electronic device 70 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 712. As shown, network adapter 712 communicates with other modules of electronic device 70 via bus 703. It should be understood that, although... Figure 8 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 70, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0217] The processing unit 701 executes various functional applications and page processing by running programs stored in the system memory 702, such as implementing the remote surgical robot image transmission adaptive control method provided in the embodiments of this application.

[0218] Computer program code for performing the operations of the embodiments of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0219] Note that the above description is merely a preferred embodiment and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this application, and the scope of this application is determined by the scope of the appended claims.

Claims

1. An adaptive control method for image transmission in a remote surgical robot, characterized in that, The method includes: Acquire the video stream captured by the endoscope during remote surgery, as well as the network status parameters of the current network link; The video stream and the network state parameters are processed in parallel to generate surgical scene perception information and network state perception information. Based on the network status perception information, the surgical scene perception information, and the preset surgical safety constraints, an adjustment scheme for the image transmission parameters is determined. According to the adjustment scheme, the image transmission parameters are adjusted, and an image transmission control signal is output.

2. The method according to claim 1, characterized in that, The parallel perception processing of the video stream and the network state parameters, outputting surgical scene perception information and network state perception information, includes: Based on the network state parameters, the low-order bandwidth estimate of the network bandwidth is determined through a time-series feature extraction model to obtain the network state perception information. Based on the keyframes in the video stream, the label of the current surgical stage is determined by an image feature extraction model to obtain the surgical scene perception information.

3. The method according to claim 2, characterized in that, The step of determining the low-order bandwidth estimate of the network bandwidth based on the network state parameters and using a time-series feature extraction model includes: The network status parameters are acquired for a preset number of collection periods to form a historical network status sequence; wherein, the network status parameters include available bandwidth, one-way latency, latency jitter, and packet loss rate; After filtering and preprocessing the historical network state sequence, the time dependency of the network state parameters is extracted through a lightweight temporal convolutional network to determine the low-order bandwidth estimate of the available bandwidth within the future time window.

4. The method according to claim 2, characterized in that, The step of determining the current surgical stage label based on keyframes in the video stream using an image feature extraction model includes: Keyframes are extracted from the video stream based on a preset time interval; The keyframes are input into a lightweight convolutional neural network to extract instrument and tissue features from the keyframes, thereby determining the probability of each surgical stage to which each keyframe belongs, and the surgical stage label with the highest probability is determined as the current surgical stage label.

5. The method according to claim 4, characterized in that, The process of inputting the keyframes into a lightweight convolutional neural network to extract instrument and tissue features from the keyframes, determining the probability of each surgical stage to which the current keyframe belongs, and identifying the surgical stage label with the highest probability as the current surgical stage label includes: The observation sequence is formed by continuously acquiring multiple recent keyframes; Each frame in the observation sequence is input into the lightweight convolutional neural network to obtain the posterior probability vector of each surgical stage to which each key frame belongs; The posterior probability vectors of all keyframes in the observation sequence are accumulated and normalized. The surgical stage label with the highest cumulative total probability is determined as the current surgical stage label.

6. The method according to claim 1, characterized in that, The step of determining the image transmission parameter adjustment scheme based on the network state awareness information, the surgical scene awareness information, and preset surgical safety constraints includes: Based on the low-bit bandwidth estimate in the network state awareness information, the upper limit constraint of the code rate is determined. Based on the surgical stage labels in the surgical scene perception information, the allocation method of resolution weight and frame rate weight in the image quality optimization target is determined. Based on the preset surgical safety constraints, the upper limit of bit rate constraints, and the allocation method, the parameter combination that satisfies the constraints and has the highest image quality score is selected from multiple candidate parameter combinations as the adjustment scheme for image transmission parameters.

7. The method according to claim 6, characterized in that, The step of selecting the parameter combination that satisfies the constraints and has the highest image quality score from multiple candidate parameter combinations based on the preset surgical safety constraints, the bitrate upper limit constraint, and the allocation method includes: Obtain the preset surgical safety constraints; wherein, the preset surgical safety constraints include at least one of the following: end-to-end latency constraints, image quality baseline constraints, buffer size and jitter matching constraints, and forward error correction redundancy rate dynamic adjustment constraints; Under the premise of simultaneously satisfying the surgical safety constraints and the bit rate upper limit constraints, the overall image quality score of each candidate parameter combination is calculated according to the allocation method, and the candidate parameter combination with the highest overall image quality score is determined as the optimal image transmission parameter adjustment scheme.

8. The method according to claim 7, characterized in that, Under the premise of simultaneously satisfying the surgical safety constraints and the bitrate upper limit constraints, the image quality comprehensive score of each candidate parameter combination is calculated according to the allocation method, and the candidate parameter combination with the highest image quality comprehensive score is determined as the optimal image transmission parameter adjustment scheme, including: Based on the surgical safety constraints and the bitrate upper limit constraints, the candidate parameter combinations are pruned to exclude those that do not meet any of the constraints. Based on the remaining candidate parameter combinations, the resolution score, frame rate score, and image stability score for each candidate parameter combination are determined respectively. Based on the resolution weight and frame rate weight corresponding to the surgical stage label in the surgical scene perception information, the resolution score, frame rate score and image stability score are weighted and summed to obtain the comprehensive image quality score of each candidate parameter combination. Compare the overall image quality scores of each candidate parameter combination, and select the candidate parameter combination with the highest score as the optimal image transmission parameter adjustment scheme.

9. The method according to claim 8, characterized in that, The process of pruning candidate parameter combinations based on the surgical safety constraints and the bitrate upper limit constraints, eliminating candidate parameter combinations that do not meet either constraint, includes: Based on the image quality baseline constraint in the surgical safety constraints, candidate parameter combinations with resolution, frame rate, or bit rate lower than their respective minimum thresholds are excluded. Based on the end-to-end delay constraint in the surgical safety constraints, candidate parameter combinations that exceed the end-to-end delay threshold are excluded. Based on the buffer size and jitter matching constraints in the surgical safety constraints, candidate parameter combinations that do not match the buffer size with the filtered delay jitter or whose buffer size exceeds the preset range are excluded. Based on the aforementioned upper limit constraint on bitrate, candidate parameter combinations with bitrates exceeding the upper limit are excluded.

10. The method according to claim 8, characterized in that, The image stability score is determined in the following way: Based on the frame rate variance within a preset time window and the number of stuttering frames that are continuously lower than the target frame rate, the image stability score is determined according to the preset quantization grading rules. The quantization and grading rules include: when the frame rate variance does not exceed a first preset threshold and there are no consecutive second preset number of frames or more below the target frame rate, it is determined as a first preset score; when the frame rate variance is greater than the first preset threshold but does not exceed a third preset threshold, or when there are consecutive fourth preset number of frames to fifth preset number of frames below the target frame rate and the frame rate of a single frame is not lower than a sixth preset percentage of the target frame rate, it is determined as a second preset score; when the frame rate variance is greater than a third preset threshold, or when there are consecutive seventh preset number of frames or more below the target frame rate, or the frame rate of a single frame is lower than an eighth preset percentage of the target frame rate, it is determined as a third preset score.

11. The method according to claim 1, characterized in that, The step of adjusting the image transmission parameters according to the adjustment scheme and outputting an image transmission control signal includes: Based on the resolution and frame rate in the adjustment scheme, the parameters are adjusted according to the preset progressive adjustment strategy; the bitrate adjustment is performed linearly and finely with an amplitude not exceeding a preset percentage of the current bitrate. The output contains an image transmission control signal with adjusted image transmission parameters.

12. The method according to claim 11, characterized in that, The step of adjusting parameters according to a preset gradual adjustment strategy includes: For resolution upgrade operations, the resolution will be increased by one level only when the low bandwidth estimate in the network state perception information meets the upgrade conditions for a first preset duration. For resolution degradation operations, the resolution will be reduced by one level only when the low bandwidth estimate in the network state awareness information meets the degradation condition for a second consecutive preset duration; wherein the first preset duration is greater than the second preset duration. Frame rate adjustment follows the same lag detection logic as resolution adjustment.

13. The method according to claim 1, characterized in that, The method further includes: Based on the network status parameters and the current cache occupancy rate, the probability of network transmission risk is determined by a risk prediction model to obtain the risk warning perception information. Accordingly, adjusting the image transmission parameters according to the adjustment scheme further includes: When the risk probability in the risk warning perception information exceeds the preset warning threshold, a preventive reduction operation is performed on the bitrate in the adjustment scheme.

14. The method according to claim 1, characterized in that, When multiple network transmission links exist, it also includes: Obtain the bandwidth and latency status of each available network link; Based on the bandwidth and latency status of each link, the video stream data in the adjusted image transmission parameters is allocated to the links with sufficient bandwidth, and the key frame data is allocated to the links with sufficient latency. The output includes the adjusted image transmission parameters and the image transmission control signal with the results of the multi-link allocation.

15. The method according to claim 1, characterized in that, After the output image transmission control signal, it also includes: When the deviation between the actual end-to-end transmission delay and the predicted state in the network state perception information exceeds a preset deviation threshold within a consecutive preset number of monitoring cycles, the model perception processing is deemed to have failed. In response to the determination, the processing mode in the parallel sensing process is disabled, and the network state parameters are processed in a pure rule mode based on filtering and linear prediction.

16. An adaptive control device for image transmission in a remote surgical robot, characterized in that, The device includes: The data acquisition module is used to acquire the video stream captured by the endoscope during remote surgery, as well as the network status parameters of the current network link; The parallel perception processing module is used to perform parallel perception processing on the video stream and the network state parameters, and output surgical scene perception information and network state perception information. The constraint optimization decision module is used to determine the adjustment scheme of image transmission parameters based on the network state perception information, the surgical scene perception information, and preset surgical safety constraints. The transmission signal output module is used to adjust the image transmission parameters according to the adjustment scheme and output an image transmission control signal.

17. An electronic device, characterized in that, Electronic devices include: At least one processor; and A memory that is communicatively connected to at least one processor; wherein, The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the remote surgical robot image transmission adaptive control method of any one of claims 1-15.

Images