Image message qos policy determination method and device, electronic equipment, storage medium and program product

Abstract

This application provides a method, apparatus, electronic device, storage medium, and program product for determining the QoS strategy of image packets, relating to the field of data transmission technology. By determining the QoS value through a first frequency domain filter and a second frequency domain filter, it achieves finer-grained monitoring of image packet transmission quality in teleoperation surgery, improving the effectiveness of image quality monitoring in teleoperation surgery. This application delves into the image information level for in-depth analysis and service monitoring, which helps improve the accuracy of determining the optimal QoS strategy. By determining the maximum QoS value based on a set number of QoS values, the optimal QoS strategy is obtained, providing the optimal image packet forwarding path for teleoperation surgery. The image packet QoS strategy determination method of this application requires low investment and has good stability, solving the problems of high investment and severe packet loss in image packet quality monitoring during teleoperation surgery.

Description

Technical Field

[0001] This application relates to the field of data transmission technology, and in particular to a method, apparatus, electronic device, storage medium, and program product for determining the QoS policy of image packets. Background Technology

[0002] Since the widespread adoption of 5G (5th Generation Mobile Communication technology) in 2020, its high bandwidth, low latency, and massive access capabilities have significantly expanded application scenarios in vertical industries. Computer Vision (CV) refers to the ability to process, analyze, recognize, and understand digital images or videos using computer and digital signal processing technologies. 5G and CV technologies are particularly well-suited to the real-time, low-latency, precise remote operation and ultra-high bandwidth wireless access requirements of remotely operated surgical robots. In practical remote surgical applications, surgeons perform surgical operations on robotic arms at a remote operating platform thousands of miles away via remote video and operating systems. Whether or not packet loss occurs during long-distance transmission of image messages is one of the factors determining the success of the remote surgical operation.

[0003] Currently, there are three methods for monitoring image quality after remote transmission of data from a teleoperated surgical robot under a 5G network: equipment performance and fault alarm monitoring method, monitoring method based on open-source data communication software, and service quality monitoring method based on a deep packet analysis system.

[0004] Equipment performance and fault alarm monitoring methods cannot detect fine-grained service fluctuations. Monitoring methods based on open-source data communication software have limited monitoring scope and cannot detect image information service dimensions. Service quality monitoring methods based on deep packet analysis systems require huge investments and suffer from severe packet loss.

[0005] In summary, the existing methods for image quality monitoring in remote surgical procedures are ineffective. Summary of the Invention

[0006] This application provides a method, apparatus, electronic device, storage medium, and program product for determining the QoS policy of image messages, in order to solve the defects of poor image quality monitoring in existing technologies for remote surgery, and to improve the image quality monitoring effect of remote surgery.

[0007] In a first aspect, this application provides a method for determining the QoS strategy of image packets, comprising: acquiring a first frequency domain filter for the image packet at the current moment during a teleoperation operation and a second frequency domain filter for the previous moment adjacent to the current moment, wherein the first frequency domain filter is determined based on the input image matrix of the image packet; determining the QoS value of the input image matrix at the current moment based on the first and second frequency domain filters, and selecting the maximum QoS value from a set number of QoS values; and acquiring the optimal QoS strategy for the image packet based on the maximum QoS value, wherein the optimal QoS strategy characterizes the best conditions for image packet transmission.

[0008] In one embodiment, the first frequency domain filter is determined based on the following steps: obtaining the output image matrix corresponding to the input image matrix in the time domain based on the time domain filter; performing a fast Fourier transform on the input image matrix to obtain a frequency domain input matrix, and performing a fast Fourier transform on the output image matrix to obtain a frequency domain output matrix; determining the filter numerator of the first frequency domain filter based on the current frequency domain input matrix, the complex conjugate matrix of the current frequency domain output matrix, the learning rate of the current and previous times, and the filter numerator of the second frequency domain filter; determining the filter denominator of the first frequency domain filter based on the current frequency domain input matrix, the complex conjugate matrix of the current frequency domain input matrix, the learning rate, and the filter denominator of the second frequency domain filter; and obtaining the first frequency domain filter based on the ratio of the filter numerator to the filter denominator of the first frequency domain filter.

[0009] In one embodiment, determining the Quality of Service (Qos) value of the input image matrix at the current time based on a first frequency domain filter and a second frequency domain filter includes: obtaining the difference between each first filter value in the first frequency domain filter and the corresponding second filter value in the second frequency domain filter; and determining the Qos value of the input image matrix based on all differences and the learning rate.

[0010] In one embodiment, after obtaining the optimal QoS policy for image packets based on the maximum QoS value, the method further includes: sending the optimal QoS policy to a signaling message management module of the 5G network QoS control platform; so that the user plane function module of the 5G network system can control the transmission of image packets from the surgical robot operating area to the surgical robot console and the proximal surgical display system based on the optimal QoS policy; wherein, the signaling message management module is used to send the optimal QoS policy to the data gateway of the 5G network QoS control platform, the data gateway is used to send the optimal QoS policy to the policy control function module of the 5G network system, the policy control function module is used to forward the optimal QoS policy to the session management function module of the 5G network system, and the session management function module is used to convert the optimal QoS policy into a protocol message and send the protocol message to the user plane function module so that the user plane function module can obtain the optimal QoS policy.

[0011] In one embodiment, the surgical robot console is started based on the following steps: the signaling message management module sends a start command to the data gateway, so that the data gateway can send the start command to the surgical robot console, and the start command is used to start the surgical robot console.

[0012] In one embodiment, the 5G network QoS control platform includes a presentation layer, a service and analysis layer, an interface layer, and a protocol layer: the presentation layer is used to display the optimal QoS strategy, the operational status of the services of the 5G network QoS control platform, and the microservice resources of the 5G network QoS control platform; the service and analysis layer is used to orchestrate, modify, and delete the services of the 5G network QoS control platform; the interface layer manages the interfaces of the 5G network QoS control platform; and the protocol layer is used to manage the communication protocols of the 5G network QoS control platform.

[0013] Secondly, this application provides an image packet QoS strategy determination apparatus, comprising: a frequency domain filter determination module, configured to acquire a first frequency domain filter of the image packet at the current moment during teleoperation surgery and a second frequency domain filter of the previous moment adjacent to the current moment, wherein the first frequency domain filter is determined based on the input image matrix of the image packet; a maximum QoS value determination module, configured to determine the QoS value of the input image matrix at the current moment based on the first and second frequency domain filters, and select the maximum QoS value from a set number of QoS values; and an optimal QoS strategy determination module, configured to acquire the optimal QoS strategy of the image packet based on the maximum QoS value, wherein the optimal QoS strategy characterizes the best conditions for image packet transmission.

[0014] Thirdly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a QoS policy determination method for any of the above-described image messages.

[0015] Fourthly, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a QoS policy determination method for any of the image packets described above.

[0016] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements a QoS policy determination method for any of the image messages described above.

[0017] The image packet QoS strategy determination method, apparatus, electronic device, storage medium, and program product provided in this application determine the QoS value through a first frequency domain filter and a second frequency domain filter at two adjacent time points. This enables monitoring of image packet transmission quality at a finer granularity during teleoperation surgery, improving the effectiveness of image quality monitoring in teleoperation surgery. This application determines the first frequency domain filter based on the input image matrix at the current time, allowing for in-depth analysis and service monitoring at the image information level, which helps improve the accuracy of determining the optimal QoS strategy. This application determines the maximum QoS value based on a set number of QoS values, thereby obtaining the optimal QoS strategy and providing the optimal image packet forwarding path for teleoperation surgery. The image packet QoS strategy determination method of this application requires less investment and has good stability, solving the problems of high investment and severe packet loss in image packet quality monitoring during teleoperation surgery. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts illustrating the QoS policy determination method for image packets provided in this application.

[0020] Figure 2 This is one of the structural diagrams of the 5G network service quality control system provided in this application.

[0021] Figure 3 This is the second schematic diagram of the 5G network service quality control system provided in this application.

[0022] Figure 4 This is the second flowchart illustrating the method for determining the QoS policy of image packets provided in this application.

[0023] Figure 5 This is a schematic diagram of the structure of the 5G network QoS control platform provided in this application.

[0024] Figure 6 This is a schematic diagram of the structure of the QoS policy determination device for image messages provided in this application.

[0025] Figure 7 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The following is combined with Figures 1-7 This application describes a method, apparatus, and electronic device for determining the QoS policy of image packets.

[0028] Figure 1 This is one of the flowcharts illustrating the method for determining the QoS policy of image packets provided in this application, such as... Figure 1 As shown, the method for determining the QoS policy of image packets includes steps S100 to S300, and the specific steps are as follows.

[0029] S100: Obtain the first frequency domain filter of the image message at the current moment in the teleoperation surgery and the second frequency domain filter of the previous moment adjacent to the current moment.

[0030] The first frequency domain filter is determined based on the input image matrix of the image message.

[0031] like Figure 2 As shown, the implementing entity of this application is a 5G network Quality of Service (QoS) control system, including a 5G network QoS control platform, a 5G network system, a surgical robot control area, and a surgical robot operation area. The 5G network system includes a Policy Control function (PCF), a Session Management Function (SMF), a User Plane Function (UPF), transmission network equipment, and wireless network equipment.

[0032] The 5G network QoS control platform collects video and control information from the surgical robot's control and operation areas. Simultaneously, the 5G network QoS control platform interacts with the 5G network system's policy control module to implement QoS policies. After the QoS policies are issued, the 5G network QoS control platform monitors the quality of the video images in the surgical robot's control area.

[0033] The surgical robot control area includes the surgical robot console and the proximal surgical display system. The surgical robot console serves as the surgeon's remote operating platform, utilizing joysticks, pedals, buttons, and other controls to perform remote surgical procedures. The proximal surgical display system provides the surgeon with a high-definition view of the surgical procedure, displaying the patient's windows, organs, blood vessels, the surgical robot's robotic arm, and end effectors.

[0034] The surgical robot operating area includes a remote surgical robot control panel and a remote surgical display system. The surgical robot control panel serves as the surgeon's remote operating environment, directly providing the surgical robot's robotic arms and end effectors for surgical procedures. The remote surgical display system provides surgical operation videos for remote consultations and learning.

[0035] The transmission network equipment is a 5G transmission network, providing a dedicated line access environment for the surgical robot control console. The wireless network equipment consists of 5G base stations, providing a wireless access environment for the surgical robot's operating area.

[0036] like Figure 2 As shown, the 5G network QoS control platform is connected to the surgical robot console via Interface A, which is used for the interaction of control commands (e.g., start commands) between the 5G network QoS control platform and the surgical robot console. The 5G network QoS control platform is connected to the proximal surgical display system via Interface B, which is used to collect video images of the surgical operation field of view that the surgeon can observe.

[0037] The data gateway and PCF are connected via the C interface, used by the 5G network QoS control platform to issue different QoS policies to the PCF. The PCF and SMF are connected via the N7 interface, used to convert the QoS policies issued by the 5G network QoS control platform into HTTP service-based instructions and send them to the SMF. The SMF and UPF are connected via the N4 interface. The SMF converts the QoS policies issued by the 5G network QoS control platform into Packet Forwarding Control Protocol (PFCF) signaling messages and sends them to the UPF. The UPF executes the user's QoS policies in the PFCF signaling messages issued by the SMF, performing operations such as buffering, forwarding, and discarding image packets.

[0038] like Figure 3As shown, the 5G network QoS control platform is deployed in the cloud. For example, the 5G network QoS control platform is deployed on an x86 virtual machine in the cloud based on the Ubuntu 16.04.1 LTS Xenial Xerus and ROS Kinetic Kame (robot operating system) framework. Both the surgical robot control area and the surgical robot operation area use IP bearer. The IP network between the 5G network QoS control platform and the 5G network system, the surgical robot console, and the surgical robot operation area is established through pre-configured data. The 5G network QoS control platform communicates with the PCF of the 5G network system, the surgical robot console, and the proximal surgical display system through a private IP network. The surgical robot arm and end effector in the surgical robot operation area are routed to the surgical robot console through the 5G base station, the Service Provider Network (SPN), the Access and Mobility Management Function (AMF) module of the 5G network system, the Unified Data Management (UDM) module of the 5G network system, and the UPF. The 5G base station is connected to the transmission SPN via the N2 or N3 interface, and the transmission SPN and AMF are connected via the N2 interface.

[0039] like Figure 3 As shown, during teleoperation surgery, the surgical robot's operating area generates image packets. These image packets are transmitted to the surgical robot's control console and proximal surgical display system (surgical robot control area) via the 5G transmission network and UPF. The UPF controls the transmission of image packets between the surgical robot's operating area and control area according to the QoS policy. The surgical robot's control console acquires the surgical operation field of view based on the image packets and performs the surgical operation. Simultaneously, the proximal surgical display system copies and forwards the image packets to the 5G network QoS control platform via the B interface and IP private network, allowing the 5G network QoS control platform to calculate the QoS policy for the image packets.

[0040] like Figure 4 As shown, the 5G network QoS control platform includes a data gateway, a digitized image module, an FFT transform module, a frequency domain filter generation module, a frequency filter update module, a QoS policy calculation module, an optimal QoS policy selection module, a result presentation module, and a signaling message management module.

[0041] The data gateway of the 5G network QoS control platform sends the received image packets to the digitization image module. The digitization image module performs discrete digitization processing on the image packets to obtain the input image matrix at the current moment. The digitization image module then sends the input image matrix to the FFT transform module, which performs a Fast Fourier Transform (FFT) on the input image matrix to obtain the frequency domain output matrix. The frequency domain filter generation module generates the first frequency domain filter for the current moment. The frequency filter update module updates the first frequency domain filter based on the second frequency domain filter from the previous moment.

[0042] The first frequency domain filter is determined based on the following steps: obtaining the output image matrix corresponding to the input image matrix in the time domain based on the time domain filter; performing a Fast Fourier Transform on the input image matrix to obtain the frequency domain input matrix, and performing a Fast Fourier Transform on the output image matrix to obtain the frequency domain output matrix; determining the filter numerator of the first frequency domain filter based on the current frequency domain input matrix, the complex conjugate matrix of the current frequency domain output matrix, the learning rate of the current and previous times, and the filter numerator of the second frequency domain filter; determining the filter denominator of the first frequency domain filter based on the current frequency domain input matrix, the complex conjugate matrix of the current frequency domain input matrix, the learning rate, and the filter denominator of the second frequency domain filter; and obtaining the first frequency domain filter based on the ratio of the filter numerator to the filter denominator of the first frequency domain filter.

[0043] In the time domain, the convolution calculation between the time-domain filter and the input image matrix is ​​computationally intensive and consumes significant computing resources. Therefore, the Fast Fourier Transform (FFT) is used to transform the convolution calculation in the time domain into a multiplication operation in the frequency domain, greatly reducing the computational cost. The formula for the convolution calculation between the time-domain filter and the input image matrix is ​​as follows.

[0044] (Formula 1);

[0045] in, The response output is in the time domain. Given an input image matrix, This is a time-domain filter.

[0046] Formula 1 yields the frequency domain output matrix after Fast Fourier Transform.

[0047] (Formula 2);

[0048] in, For the Fast Fourier Transform function, For frequency domain output matrix, For the frequency domain input matrix, Let be the complex conjugate matrix of the frequency domain filter.

[0049] Obtain the deviation function of the frequency domain input matrix and frequency domain output matrix under the frequency domain filter. The formula for calculating the deviation function is as follows.

[0050] (Formula 3);

[0051] in, It is the deviation function. This is the horizontal index of the frequency domain filter. This is the vertical index of the frequency domain filter. For the first A frequency domain input matrix with x and y dimensions, For the first A frequency domain output matrix with x and y dimensions. Let be the complex conjugate matrix of the frequency domain filter with xy dimensions. The number of frequency domain input matrices, Let be the total number of rows in the complex conjugate matrix of the frequency domain filter. This represents the total number of columns in the complex conjugate matrix of the frequency domain filter.

[0052] Desiring the deviation function Find the minimum value of the bias function and differentiate it. When the first partial derivative of the bias function is 0, the calculation formula for the frequency domain filter is obtained.

[0053] Formula (4);

[0054] in, This is the horizontal index of the frequency domain filter. This is the vertical index of the frequency domain filter. For the first A frequency domain input matrix with x and y dimensions, For the first A frequency domain output matrix with x and y dimensions. For the first A complex conjugate matrix of a frequency domain output matrix with x and y dimensions. For the first A complex conjugate matrix of a frequency domain input matrix with x and y dimensions. For a frequency domain filter with xy dimensions, The number of frequency domain input matrices, Let be the total number of rows in the complex conjugate matrix of the frequency domain filter. This represents the total number of columns in the complex conjugate matrix of the frequency domain filter.

[0055] The initial frequency domain filter at the current time can be calculated using Formula 4. The frequency domain filter is a function of time. Updating the initial frequency domain filter at the current time (time k) yields the first frequency domain filter.

[0056] (Formula 5);

[0057] in, For the current time (time k), the first frequency domain filter with xy dimensions, The filter molecule of the first frequency domain filter, The filter denominator for the first filter. For learning rate, Let be the i-th frequency domain input matrix with x and y dimensions at time k. Let be the complex conjugate matrix of the i-th frequency domain input matrix with xy dimensions at time k. Let be the i-th frequency domain output matrix with xy dimensions at time k. Let be the complex conjugate matrix of the i-th frequency domain output matrix with xy dimensions at time k. This is the filter molecule of the second frequency domain filter from the previous time step. This is the filter denominator for the second filter.

[0058] Furthermore, the first frequency domain filter at the start time is calculated according to Formula 4.

[0059] For formula 5, when hour, For extreme cases Completely by Decision, and the current and It's irrelevant. And when... hour, For extreme cases Completely by the current and Decision, and Irrelevant. When hour, For extreme cases Completely by Decision, and the current and It's irrelevant. And when... hour, For extreme cases Completely by the current and Decision, and Irrelevant. Clearly, when the surgical robot's operating table is in operation, changes in the lighting of the surgical environment, as well as changes in the size and posture of the surgical robot's robotic arm and end effector, allow for the acquisition of a relatively ideal first frequency domain filter at the current moment using the second frequency domain filter from the previous moment and the learning rate. This enables the frequency domain filter to quickly adapt to changes in the target's appearance while ensuring its robustness. However, when frame drops occur in the video image, a significant deviation occurs between the first and second frequency domain filters.

[0060] This application reduces computation by converting convolution operations in the time domain into multiplication operations in the frequency domain using Fast Fourier Transform. The first frequency domain filter is updated based on the second frequency domain filter and learning rate from the previous time step, ensuring both rapid adaptation of the frequency domain filter to changes in the target's appearance and its robustness.

[0061] S200: Determine the QoS value of the input image matrix at the current time based on the first frequency domain filter and the second frequency domain filter, and select the maximum QoS value from a set number of QoS values.

[0062] The Quality of Service (Qos) value of the input image matrix at the current time is determined based on the first frequency domain filter and the second frequency domain filter. Specifically, the difference between each first filter value in the first frequency domain filter and the corresponding second filter value in the second frequency domain filter is obtained; the Qos value of the input image matrix is ​​determined based on all differences and the learning rate.

[0063] The QoS policy calculation module calculates the difference between each first filter value in the first frequency domain filter and the corresponding second filter value in the second frequency domain filter, and calculates the QoS value of the input image matrix based on all differences and the learning rate. The formula for calculating the QoS value is as follows.

[0064] ;

[0065] in, This is the first frequency domain filter. This is the second frequency domain filter. Let be the first filter value in the x-th row and y-th column of the first frequency domain filter. This represents the second filter value in the x-th row and y-th column of the second frequency domain filter. For learning rate, This is the QoS value. This represents the total number of rows in the frequency domain filter. This represents the total number of columns in the frequency domain filter. It is the difference between the first filter value and the second filter value in the corresponding second frequency domain filter.

[0066] This application determines the QoS value based on all the differences between the first frequency domain filter and the second frequency domain filter, and the learning rate, thus achieving accurate calculation of the QoS value.

[0067] The 5G network QoS control platform sends a predetermined number (N) of different initial QoS policies to the UPF at different times through the signaling message management module, PCF, and SMF, so that the UPF can transmit image packets between the surgical robot's operating area and control area. Simultaneously, the 5G network QoS control platform collects image packets at different times through the proximal surgical display system. Different times correspond to different Protocol Data Unit (PDU) dialogues, and the platform calculates the QoS value of the image packets at each time, resulting in a total of N (predetermined) QoS values. These N QoS values ​​are then sorted, for example, in descending order, to obtain the maximum QoS value.

[0068] S300: Optimal QoS strategy for obtaining image packets based on the maximum QoS value.

[0069] The optimal QoS strategy characterizes the best conditions for image message transmission.

[0070] Obtain the initial QoS policy corresponding to the maximum QoS value. For example, obtain the initial QoS policy corresponding to the maximum QoS value from the PDU corresponding to the maximum QoS value. Use the initial QoS policy corresponding to the maximum QoS value as the optimal QoS policy for the image packet.

[0071] QoS policies are quality standards for network services, ensuring the reliability and performance of image packet transmission. QoS policies include image transmission metrics such as latency, bandwidth, packet loss rate, and jitter.

[0072] The image packet QoS strategy determination method provided in this application determines the QoS value through a first frequency domain filter and a second frequency domain filter at two adjacent time points. This enables monitoring of image packet transmission quality at a finer granularity during teleoperation surgery, improving the effectiveness of image quality monitoring in teleoperation surgery. This application determines the first frequency domain filter based on the input image matrix at the current time, performing in-depth analysis and service monitoring at the image information level, which helps improve the accuracy of determining the optimal QoS strategy. This application determines the maximum QoS value based on a set number of QoS values, thereby obtaining the optimal QoS strategy and providing the optimal image packet forwarding path for teleoperation surgery. The image packet QoS strategy determination method of this application requires less investment and has good stability, solving the problems of high investment and severe packet loss in image packet quality monitoring during teleoperation surgery.

[0073] Based on the above embodiments, after obtaining the optimal QoS strategy for image packets based on the maximum QoS value, step S400 is further included.

[0074] S400: The signaling message management module sends the optimal QoS policy to the 5G network QoS control platform so that the user plane function module of the 5G network system can control the transmission of image packets from the surgical robot operation area to the surgical robot console and the proximal surgical display system based on the optimal QoS policy.

[0075] The signaling message management module sends the optimal QoS policy to the data gateway of the 5G network QoS control platform. The data gateway sends the optimal QoS policy to the policy control function module of the 5G network system. The policy control function module forwards the optimal QoS policy to the session management function module of the 5G network system. The session management function module converts the optimal QoS policy into a protocol message and sends the protocol message to the user plane function module so that the user plane function module can obtain the optimal QoS policy.

[0076] like Figure 4 As shown, the optimal QoS policy selection module of the 5G network QoS control platform sends the optimal QoS policy to the signaling message management module. The signaling message management module then sends the optimal QoS policy to the policy control function module (PCF) of the 5G network system via the data gateway. The PCF returns a message indicating successful issuance of the optimal QoS policy via the data gateway. The PCF forwards the optimal QoS policy to the session management function module (SMF), and the SMF returns a message indicating successful execution to the PCF. The SMF converts the optimal QoS policy into a protocol message, such as the Packet Forwarding Control Protocol (PFCP), and issues the PFCP to the UPF via the N4 interface. The UPF then controls the transmission of image messages from the surgical robot's operating area to the surgical robot's control console and proximal surgical display system according to the optimal QoS policy. Simultaneously, the UPF returns a message indicating successful execution to the SMF.

[0077] This application distributes the optimal QoS strategy to the UPF through a data gateway, PCF, and SMF, and controls the transmission of image packets through the UPF, thereby realizing quality monitoring of the image information service dimension and improving the efficiency of image packet transmission in teleoperation surgery.

[0078] Furthermore, the surgical robot console is started based on the following steps: the signaling message management module sends a start command to the data gateway, so that the data gateway can send the start command to the surgical robot console, and the start command is used to start the surgical robot console.

[0079] like Figure 4As shown, the signaling message management module of the 5G network QoS control platform sends a start command to the surgical robot console through the data gateway. The data gateway then sends the start command to the surgical robot console. The surgical robot console responds to the start command and initiates startup. The surgical robot console returns a startup success message to the signaling message management module of the 5G network QoS control platform.

[0080] This application enables remote start-up and control of the surgical robot console by launching the start command of the signaling message management module, which helps to ensure the smooth operation of remote surgery.

[0081] Based on the above embodiments, the 5G network QoS control platform includes a presentation layer, a service and analysis layer, an interface layer, and a protocol layer.

[0082] The presentation layer is used to display the optimal QoS strategy, the operational status of the services of the 5G network QoS control platform, and the microservice resources of the 5G network QoS control platform.

[0083] The service and analytics layer is used to orchestrate, modify, and delete services on the 5G network QoS control platform.

[0084] The interface layer manages the interfaces of the 5G network QoS control platform.

[0085] The protocol layer is used to manage the communication protocols of the 5G network QoS control platform.

[0086] like Figure 5 As shown, the 5G network QoS control platform adopts a layered microservice architecture with a total of 4 layers, including the presentation layer, service and analysis layer, interface layer and protocol layer.

[0087] The presentation layer includes a microservice resource presentation unit, an optimal QoS policy presentation unit, and a service process presentation unit. The presentation unit is used to display various information of the 5G network QoS control platform, including the optimal QoS policy, the operational status of services on the 5G network QoS control platform, and the microservice resources of the 5G network QoS control platform.

[0088] The interface layer includes orchestration interface units, business interface units, and microservice interface units. The interface layer provides internal and external interface management capabilities.

[0089] The service and analytics layer includes a service orchestration management unit, a service orchestrator execution engine, and a microservice management unit. This layer is used to orchestrate, modify, and delete services on the 5G network QoS control platform.

[0090] The protocol layer includes the protocol orchestration unit and the communication protocols of the 5G network QoS control platform. These communication protocols include the Robot Operating System (ROS) protocol, Simple Network Management Protocol (SNMP), and Hypertext Transfer Protocol (HTTP), among others.

[0091] This application implements different functional management and presentation of the 5G network QoS control platform through the presentation layer, service and analysis layer, interface layer and protocol layer, which helps to improve the working efficiency of the 5G network QoS control platform.

[0092] The following describes the QoS policy determination apparatus for image packets provided in this application. The QoS policy determination apparatus for image packets described below can be referred to in correspondence with the QoS policy determination method for image packets described above.

[0093] like Figure 6 As shown, an image message QoS strategy determination device includes: a frequency domain filter determination module 601, used to obtain a first frequency domain filter of the image message at the current moment in teleoperation surgery and a second frequency domain filter of the previous moment adjacent to the current moment, wherein the first frequency domain filter is determined based on the input image matrix of the image message.

[0094] The maximum QoS value determination module 602 is used to determine the QoS value of the input image matrix at the current time based on the first frequency domain filter and the second frequency domain filter, and select the maximum QoS value from a set number of QoS values.

[0095] The optimal QoS strategy determination module 603 is used to obtain the optimal QoS strategy for image packets based on the maximum QoS value. The optimal QoS strategy represents the best conditions for image packet transmission.

[0096] The image packet QoS strategy determination device provided in this application determines the QoS value through a first frequency domain filter and a second frequency domain filter at two adjacent time points, realizing the monitoring of image packet transmission quality at a finer granularity in teleoperation surgery and improving the effectiveness of image quality monitoring in teleoperation surgery. This application determines the first frequency domain filter based on the input image matrix at the current time, performing in-depth analysis and service monitoring at the image information level, which helps improve the accuracy of determining the optimal QoS strategy. This application determines the maximum QoS value based on a set number of QoS values, thereby obtaining the optimal QoS strategy and providing the optimal image packet forwarding path for teleoperation surgery. The image packet QoS strategy determination method of this application requires less investment and has good stability, solving the problems of high investment and severe packet loss in image packet quality monitoring during teleoperation surgery.

[0097] In one embodiment, the frequency domain filter determination module 601 is configured to: obtain the output image matrix corresponding to the input image matrix in the time domain based on the time domain filter; perform a fast Fourier transform on the input image matrix to obtain a frequency domain input matrix, and perform a fast Fourier transform on the output image matrix to obtain a frequency domain output matrix; determine the filter numerator of the first frequency domain filter based on the frequency domain input matrix at the current time, the complex conjugate matrix of the frequency domain output matrix at the current time, the learning rate at the current time and the previous time, and the filter numerator of the second frequency domain filter; determine the filter denominator of the first frequency domain filter based on the frequency domain input matrix at the current time, the complex conjugate matrix of the frequency domain input matrix at the current time, the learning rate, and the filter denominator of the second frequency domain filter; and obtain the first frequency domain filter based on the ratio of the filter numerator and the filter denominator of the first frequency domain filter.

[0098] In one embodiment, the maximum QoS value determination module 602 is used to: obtain the difference between each first filter value in the first frequency domain filter and the corresponding second filter value in the second frequency domain filter; and determine the QoS value of the input image matrix based on all differences and the learning rate.

[0099] In one embodiment, the optimal QoS policy determination module 603 is further configured to: send the optimal QoS policy to the signaling message management module of the 5G network QoS control platform; so that the user plane function module of the 5G network system can control the transmission of image packets from the surgical robot operating area to the surgical robot console and the proximal surgical display system based on the optimal QoS policy; wherein, the signaling message management module is configured to send the optimal QoS policy to the data gateway of the 5G network QoS control platform, the data gateway is configured to send the optimal QoS policy to the policy control function module of the 5G network system, the policy control function module is configured to forward the optimal QoS policy to the session management function module of the 5G network system, and the session management function module is configured to convert the optimal QoS policy into a protocol message and send the protocol message to the user plane function module so that the user plane function module can obtain the optimal QoS policy.

[0100] In one embodiment, the optimal QoS strategy determination module 603 is further configured to: send a start command to the data gateway based on the signaling message management module, so that the data gateway can issue the start command to the surgical robot console, and the start command is used to start the surgical robot console.

[0101] In one embodiment, the 5G network QoS control platform includes a presentation layer, a service and analysis layer, an interface layer, and a protocol layer: the presentation layer is used to display the optimal QoS strategy, the operational status of the services of the 5G network QoS control platform, and the microservice resources of the 5G network QoS control platform; the service and analysis layer is used to orchestrate, modify, and delete the services of the 5G network QoS control platform; the interface layer manages the interfaces of the 5G network QoS control platform; and the protocol layer is used to manage the communication protocols of the 5G network QoS control platform.

[0102] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740. The processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a QoS policy determination method for image packets. This method includes: acquiring a first frequency domain filter for the image packet at the current moment during teleoperation and a second frequency domain filter for the previous moment adjacent to the current moment; the first frequency domain filter is determined based on the input image matrix of the image packet; determining the QoS value of the input image matrix at the current moment based on the first and second frequency domain filters; selecting the maximum QoS value from a set number of QoS values; and obtaining the optimal QoS policy for the image packet based on the maximum QoS value.

[0103] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0104] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image packet QoS strategy determination method provided by the above methods. The method includes: acquiring a first frequency domain filter for the image packet at the current moment in a teleoperation operation and a second frequency domain filter for the previous moment adjacent to the current moment, wherein the first frequency domain filter is determined based on the input image matrix of the image packet; determining the QoS value of the input image matrix at the current moment based on the first and second frequency domain filters, selecting the maximum QoS value from a set number of QoS values; and acquiring the optimal QoS strategy for the image packet based on the maximum QoS value.

[0105] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for determining the QoS strategy of image packets provided by the methods described above. The method includes: acquiring a first frequency domain filter for the image packet at the current moment during teleoperation surgery and a second frequency domain filter for the previous moment adjacent to the current moment, wherein the first frequency domain filter is determined based on the input image matrix of the image packet; determining the QoS value of the input image matrix at the current moment based on the first and second frequency domain filters, selecting the maximum QoS value from a set number of QoS values; and acquiring the optimal QoS strategy for the image packet based on the maximum QoS value.

[0106] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0107] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for determining the QoS policy of image packets, characterized in that, include: The first frequency domain filter of the image message at the current moment in the teleoperation surgery and the second frequency domain filter of the previous moment adjacent to the current moment are obtained. The first frequency domain filter is determined based on the input image matrix of the image message. The QoS value of the input image matrix at the current time is determined based on the first frequency domain filter and the second frequency domain filter, and the maximum QoS value is selected from a set number of QoS values. The optimal QoS strategy for the image packet is obtained based on the maximum QoS value, and the optimal QoS strategy represents the best conditions for the transmission of the image packet. The first frequency domain filter is determined based on the following steps: The output image matrix corresponding to the input image matrix in the time domain is obtained based on the time domain filter; The input image matrix is ​​subjected to a Fast Fourier Transform to obtain a frequency domain input matrix, and the output image matrix is ​​subjected to a Fast Fourier Transform to obtain a frequency domain output matrix. The filter molecule of the first frequency domain filter is determined based on the current frequency domain input matrix, the complex conjugate matrix of the current frequency domain output matrix, the learning rate of the current time and the previous time, and the filter molecule of the second frequency domain filter. The filter denominator of the first frequency domain filter is determined based on the frequency domain input matrix at the current time, the complex conjugate matrix of the frequency domain input matrix at the current time, the learning rate, and the filter denominator of the second frequency domain filter. The first frequency domain filter is obtained by the ratio of the filter numerator of the first frequency domain filter to the filter denominator of the first frequency domain filter.

2. The method for determining the QoS strategy of image packets according to claim 1, characterized in that, The step of determining the Quality of Service (QoS) value of the input image matrix at the current time based on the first frequency domain filter and the second frequency domain filter includes: Obtain the difference between each first filter value in the first frequency domain filter and the corresponding second filter value in the second frequency domain filter; The Qos value of the input image matrix is ​​determined based on all the differences and the learning rate.

3. The method for determining the QoS strategy of image packets according to claim 1, characterized in that, After obtaining the optimal QoS strategy for the image packet based on the maximum QoS value, the method further includes: The optimal QoS strategy is sent to the signaling message management module of the 5G network QoS control platform so that the user plane function module of the 5G network system can control the transmission of the image packets from the surgical robot operating area to the surgical robot console and the proximal surgical display system based on the optimal QoS strategy. Specifically, the signaling message management module sends the optimal QoS policy to the data gateway of the 5G network QoS control platform. The data gateway sends the optimal QoS policy to the policy control function module of the 5G network system. The policy control function module forwards the optimal QoS policy to the session management function module of the 5G network system. The session management function module converts the optimal QoS policy into a protocol message and sends the protocol message to the user plane function module so that the user plane function module can obtain the optimal QoS policy.

4. The method for determining the QoS strategy of image packets according to claim 3, characterized in that, The surgical robot console is activated based on the following steps: The signaling message management module sends a start command to the data gateway, which then sends the start command to the surgical robot console. The start command is used to start the surgical robot console.

5. The method for determining the QoS strategy of image packets according to claim 3, characterized in that, The 5G network QoS control platform includes a presentation layer, a service and analysis layer, an interface layer, and a protocol layer. The presentation layer is used to display the optimal QoS strategy, the operation status of the services of the 5G network QoS control platform, and the microservice resources of the 5G network QoS control platform. The service and analysis layer is used to orchestrate, modify, and delete services of the 5G network QoS control platform; The interface layer manages the interfaces of the 5G network QoS control platform; The protocol layer is used to manage the communication protocols of the 5G network QoS control platform.

6. A QoS policy determination device for image packets, characterized in that, include: A frequency domain filter determination module is used to obtain a first frequency domain filter of the image message at the current moment during teleoperation surgery and a second frequency domain filter of the previous moment adjacent to the current moment. The first frequency domain filter is determined based on the input image matrix of the image message. The maximum QoS value determination module is used to determine the QoS value of the input image matrix at the current time based on the first frequency domain filter and the second frequency domain filter, and select the maximum QoS value from a set number of QoS values; The optimal QoS strategy determination module is used to obtain the optimal QoS strategy for the image packet based on the maximum QoS value, wherein the optimal QoS strategy characterizes the best conditions for the transmission of the image packet. The first frequency domain filter is determined based on the following steps: The output image matrix corresponding to the input image matrix in the time domain is obtained based on the time domain filter; The input image matrix is ​​subjected to a Fast Fourier Transform to obtain a frequency domain input matrix, and the output image matrix is ​​subjected to a Fast Fourier Transform to obtain a frequency domain output matrix. The filter molecule of the first frequency domain filter is determined based on the current frequency domain input matrix, the complex conjugate matrix of the current frequency domain output matrix, the learning rate of the current time and the previous time, and the filter molecule of the second frequency domain filter. The filter denominator of the first frequency domain filter is determined based on the frequency domain input matrix at the current time, the complex conjugate matrix of the frequency domain input matrix at the current time, the learning rate, and the filter denominator of the second frequency domain filter. The first frequency domain filter is obtained by the ratio of the filter numerator of the first frequency domain filter to the filter denominator of the first frequency domain filter.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the QoS policy determination method for image packets as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the QoS policy determination method for image packets as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the QoS policy determination method for image packets as described in any one of claims 1 to 5.

Images