Image message Qos strategy determination method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a Qos strategy determination method and device for an image message, electronic equipment, a storage medium and a program product, and relates to the technical field of data transmission, a Qos value is determined through a first frequency domain filter and a second frequency domain filter, the transmission quality of an image message with a smaller granularity level in a teleoperation surgery is monitored, and the teleoperation efficiency is improved. And the image quality monitoring effect in the teleoperation surgery is improved. According to the method, deep analysis and service monitoring are carried out in the image information level, and the accuracy of determining the optimal Qos strategy can be improved. The maximum Qos value is determined according to the set number of Qos values, then the optimal Qos strategy is obtained, and the optimal image message forwarding path is provided for the teleoperation surgery. The Qos strategy determination method for the image message is low in investment and good in stability, and solves the problems of high investment and serious packet loss of image message quality monitoring in a teleoperation surgery.

Description

Technical Field

[0001] The present application relates to the technical field of data transmission, and in particular to a method, device, electronic device, storage medium and program product for determining a QoS strategy for an image message. Background Art

[0002] Since the fifth generation mobile communication technology (5G) was widely used in 2020, its technical characteristics of large bandwidth, low latency, and massive access have greatly expanded the application scenarios of vertical industries. Computer Vision (CV) refers to the ability to process, analyze, identify, and understand digital images or videos using computers and digital signal processing technology. 5G and CV technologies are particularly suitable for the ultra-large bandwidth business scenario requirements of real-time, low-latency, precise teleoperation and wireless access of remote surgical robots. In actual teleoperated surgical applications, surgeons perform surgical operations on the robotic arms of the operating table thousands of miles away through remote video and operating systems. Whether the image message is lost during long-distance transmission will be one of the factors that determine whether the teleoperated surgery can be successful.

[0003] At present, there are three methods for monitoring image quality after remote transmission of teleoperated surgical robots under 5G networks, namely, equipment performance and fault alarm monitoring method, monitoring method based on data communication open source software, and service quality monitoring method based on deep message analysis system.

[0004] The equipment performance and fault alarm monitoring methods cannot monitor small-scale business fluctuations. The monitoring methods based on data communication open source software have a limited monitoring range and cannot detect the image information business dimension. The business quality monitoring methods based on the deep message analysis system require huge investment and suffer from serious packet loss.

[0005] In summary, the image quality monitoring in existing teleoperation surgery is poor. Summary of the invention

[0006] The present application provides a method, device, electronic device, storage medium and program product for determining the QoS strategy of an image message, so as to solve the defect of poor image quality monitoring effect of teleoperated surgery in the prior art and improve the effect of image quality monitoring of teleoperated surgery.

[0007] In a first aspect, the present application provides a method for determining the QoS strategy of an image message, comprising: obtaining a first frequency domain filter of an image message at a current moment in a teleoperated surgery and a second frequency domain filter at a previous moment adjacent to the current moment, the first frequency domain filter being determined based on an input image matrix of the image message; determining a quality of service (QoS) value of the input image matrix at the current moment based on the first frequency domain filter and the second frequency domain filter, and selecting a maximum QoS value from a set number of QoS values; obtaining an optimal QoS strategy for the image message based on the maximum QoS value, the optimal QoS strategy representing optimal conditions for image message transmission.

[0008] In one embodiment, the first frequency domain filter is determined based on the following steps: obtaining an output image matrix corresponding to an input image matrix in the time domain based on a time domain filter; performing a fast Fourier transform on the input image matrix to obtain a frequency domain input matrix, and performing a fast Fourier transform on the output image matrix to obtain a frequency domain output matrix; determining a filter numerator of the first frequency domain filter based on the frequency domain input matrix at the current moment, the complex conjugate matrix of the frequency domain output matrix at the current moment, the learning rates at the current moment and the previous moment, and the filter numerator of the second frequency domain filter; determining a filter denominator of the first frequency domain filter based on the frequency domain input matrix at the current moment, the complex conjugate matrix of the frequency domain input matrix at the current moment, the learning rate, and the filter denominator of the second frequency domain filter; obtaining the first frequency domain filter based on the ratio of the filter numerator of the first frequency domain filter to the filter denominator of the first frequency domain filter.

[0009] In one embodiment, the quality of service (Qos) value of the input image matrix at the current moment is determined based on the first frequency domain filter and the second frequency domain filter, including: obtaining the difference between each first filter value in the first frequency domain filter and the corresponding second filter value in the second frequency domain filter; determining the Qos value of the input image matrix based on all differences and the learning rate.

[0010] In one embodiment, after obtaining the optimal Qos strategy for the image message based on the maximum Qos value, it also includes: sending the optimal Qos strategy to the signaling message management module of the 5G network Qos control platform; so that the user plane function module of the 5G network system controls the transmission of the image message from the surgical robot operation area to the surgical robot console and the proximal surgical display system based on the optimal Qos strategy; wherein the signaling message management module is used to send the optimal Qos strategy to the data gateway of the 5G network Qos control platform, the data gateway is used to send the optimal Qos strategy to the policy control function module of the 5G network system, the policy control function module is used to forward the optimal Qos strategy to the session management function module of the 5G network system, the session management function module is used to convert the optimal Qos strategy into a protocol message, and send the protocol message to the user plane function module, so that the user plane function module can obtain the optimal Qos strategy.

[0011] In one embodiment, the surgical robot console is started based on the following steps: a start instruction is sent to a data gateway based on a signaling message management module, so that the data gateway sends the start instruction to the surgical robot console, and the start instruction is used to start the surgical robot console.

[0012] In one embodiment, the 5G network QoS control platform includes a presentation layer, a service and analysis layer, an interface layer and a protocol layer: the presentation layer is used to display the optimal QoS strategy, the operation status of the services of the 5G network QoS control platform and the microservice resources of the 5G network QoS control platform; the service and analysis layer is used to orchestrate, modify and delete the services of the 5G network QoS control platform; the interface layer manages the interface of the 5G network QoS control platform; the protocol layer is used to manage the communication protocol of the 5G network QoS control platform.

[0013] In a second aspect, the present application provides a device for determining a Qos strategy for an image message, comprising: a frequency domain filter determination module, used to obtain a first frequency domain filter of an image message at a current moment in a teleoperated surgery and a second frequency domain filter at a previous moment adjacent to the current moment, the first frequency domain filter being determined based on an input image matrix of the image message; a maximum Qos value determination module, used to determine the service quality Qos value of the input image matrix at the current moment based on the first frequency domain filter and the second frequency domain filter, and select the maximum Qos value from a set number of Qos values; an optimal Qos strategy determination module, used to obtain an optimal Qos strategy for the image message based on the maximum Qos value, the optimal Qos strategy representing the best conditions for image message transmission.

[0014] In a third aspect, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a method for determining a QoS policy for any of the above-mentioned image messages is implemented.

[0015] In a fourth aspect, the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for determining a QoS policy for any of the above-mentioned image messages.

[0016] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements a QoS policy determination method for any of the above-mentioned image messages.

[0017] The method, device, electronic device, storage medium and program product for determining the Qos strategy of image messages provided by the present application determine the Qos value through the first frequency domain filter and the second frequency domain filter at two adjacent moments, thereby realizing the monitoring of the transmission quality of image messages at a smaller granularity level in teleoperated surgery, and improving the effect of image quality monitoring in teleoperated surgery. The present application determines the first frequency domain filter according to the input image matrix at the current moment, and conducts in-depth analysis and business monitoring at the image information level, which is conducive to improving the accuracy of determining the optimal Qos strategy. The present application determines the maximum Qos value according to a set number of Qos values, and then obtains the optimal Qos strategy, realizing the provision of the optimal image message forwarding path for teleoperated surgery. The Qos strategy determination method for image messages of the present application requires little investment and has good stability, which solves the problem of large investment and serious packet loss in image message quality monitoring in teleoperated surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is one of the flow charts of the method for determining the QoS strategy of the image message provided in this application.

[0020] Figure 2 This is one of the structural diagrams of the 5G network service quality control system provided in this application.

[0021] Figure 3 This is the second structural diagram of the 5G network service quality control system provided by this application.

[0022] Figure 4 This is the second flow chart of the method for determining the QoS strategy of image messages provided in this application.

[0023] Figure 5 It is a structural diagram of the 5G network QoS control platform provided by this application.

[0024] Figure 6 It is a structural diagram of the QoS strategy determination device for image messages provided in this application.

[0025] Figure 7 It is a schematic diagram of the structure of the electronic device provided by this application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] Combine the following Figure 1-Figure 7 The present invention describes a method, device and electronic device for determining the QoS strategy of an image message.

[0028] Figure 1 This is one of the flow charts of the method for determining the QoS strategy of the image message provided by this application, such as Figure 1 As shown, the method for determining the QoS policy of the image message includes steps S100 to S300, and each step is specifically as follows.

[0029] S100: Acquire a first frequency domain filter of an image message at a current moment in a teleoperation surgery and a second frequency domain filter at a previous moment adjacent to the current moment.

[0030] The first frequency domain filter is determined based on an input image matrix of the image message.

[0031] like Figure 2 As shown, the execution subject of this application is a 5G network quality of service (Quality of Service, QoS) control system, including a 5G network QoS control platform, a 5G network system, a surgical robot control area, and a surgical robot operation area. The 5G network system includes a policy control function module (Policy Control function, PCF), a session management function module (Session Management Function, SMF), a user plane function module (User Plane Function, UPF), a transmission network device, and a wireless network device.

[0032] The 5G network QoS control platform collects video information and control information from the surgical robot control area and the surgical robot operation area. At the same time, the 5G network QoS control platform interacts with the policy control function module of the 5G network system to implement QoS policies. After the QoS policy is issued, the 5G network QoS control platform monitors the quality of the video images in the surgical robot control area.

[0033] The surgical robot control area includes the surgical robot console and the proximal surgical display system. The surgical robot console is the operating platform for the surgeon to remotely operate the operation, and the remote operation is performed using the joystick, pedals, buttons, etc. of the surgical robot console. The proximal surgical display system provides the surgeon with a high-definition field of view for the surgical operation, in which the patient's window, organs, blood vessels, surgical robot arm and end effector are presented.

[0034] The surgical robot operation area includes a remote surgical machine console and a remote surgical display system. The surgical machine console is the remote operating environment for surgeons, directly providing the surgical robot arm and end effector for surgical operations. The remote surgical display system provides surgical operation videos for remote consultation and learning.

[0035] The transmission network equipment is a 5G transmission network, which provides a dedicated line access environment for the surgical robot console. The wireless network equipment is composed of 5G base stations, which provide a wireless access environment for the surgical robot operation area.

[0036] like Figure 2 As shown, the 5G network QoS control platform is connected to the surgical robot console via the A interface, which is used for the interaction of control instructions (e.g., start instructions) between the 5G network QoS control platform and the surgical robot console. The 5G network QoS control platform is connected to the proximal surgical display system via the B interface, which is used to collect video images of the surgical operation field that the surgeon can observe.

[0037] The data gateway and PCF are connected through the C interface, which is used for the 5G network QoS control platform to send different QoS policies to PCF. PCF and SMF are connected through the N7 interface, which is used to convert the QoS policy sent by the 5G network QoS control platform into HTTP service instructions and send them to SMF. SMF and UPF are connected through the N4 interface. SMF converts the QoS policy sent by the 5G network QoS control platform into a Packet Forwarding Control Protocol (PFCF) signaling message and sends it to UPF. UPF executes the user's QoS policy in the PFCP signaling message sent by SMF, and performs operations such as caching, forwarding, and discarding image messages.

[0038] like Figure 3As shown in the figure, the 5G network QoS control platform is located in the cloud. For example, the 5G network QoS control platform is deployed on the cloud x86 virtual machine based on Ubuntu 16.04.1 LTS Xenial Xerus and ROS Kinetic Kame (Robot Operating System) framework. The surgical robot control area and the surgical robot operation area are both IP-bearing. The IP network between the 5G network QoS control platform and the 5G network system, the surgical robot console and the surgical robot operation area is connected by pre-configuring relevant data. The 5G network QoS control platform and the PCF, surgical robot console and proximal surgical display system of the 5G network system are interconnected through an IP private network. The surgical robot arm and end effector in the surgical robot operation area are routed to the surgical robot console through the 5G base station, the transmission service provider network (SPN), the access and mobility management function (AMF) module of the 5G network system, the unified data management (UDM) module of the 5G network system and the UPF. Among them, the 5G base station is connected to the transmission SPN through the N2 interface or the N3 interface, and the transmission SPN and AMF are connected through the N2 interface.

[0039] like Figure 3 As shown in the figure, during the teleoperation surgery, the surgical robot operation area generates image messages, which are then transmitted to the surgical robot console and the proximal surgical display system (surgical robot control area) through the 5G transmission network and UPF. Among them, UPF controls the transmission of image messages between the surgical robot operation area and the surgical robot control area according to the QoS strategy. The surgical robot console obtains the surgical operation field of view based on the image message and performs the surgical operation. At the same time, the proximal surgical display system copies and forwards the image message to the 5G network QoS control platform through the B interface and the IP private network, so that the 5G network QoS control platform can calculate the QoS strategy of the image message.

[0040] like Figure 4 As shown, the 5G network QoS control platform includes a data gateway, a digital image module, an FFT transformation module, a frequency domain filter generation module, a frequency filter update module, a QoS strategy calculation module, an optimal QoS strategy selection module, a result presentation module and a signaling message management module.

[0041] The data gateway of the 5G network QoS control platform sends the received image message to the digital image module. The digital image module performs discrete digital image processing on the image message to obtain the input image matrix at the current moment. The digital image module sends the input image matrix to the FFT transformation module, which performs Fast Fourier Transform (FFT) on the input image matrix to obtain the frequency domain output matrix. The frequency domain filter generation module generates the first frequency domain filter at the current moment. The frequency filter update module updates the first frequency domain filter according to the second frequency domain filter at the previous moment.

[0042] The first frequency domain filter is determined based on the following steps: obtaining an output image matrix corresponding to an input image matrix in the time domain based on a time domain filter; performing a fast Fourier transform on the input image matrix to obtain a frequency domain input matrix, and performing a fast Fourier transform on the output image matrix to obtain a frequency domain output matrix; determining the filter numerator of the first frequency domain filter based on the frequency domain input matrix at the current moment, the complex conjugate matrix of the frequency domain output matrix at the current moment, the learning rates at the current moment and the previous moment, and the filter numerator of the second frequency domain filter; determining the filter denominator of the first frequency domain filter based on the frequency domain input matrix at the current moment, the complex conjugate matrix of the frequency domain input matrix at the current moment, the learning rate, and the filter denominator of the second frequency domain filter; obtaining the first frequency domain filter based on the ratio of the filter numerator of the first frequency domain filter and the filter denominator of the first frequency domain filter.

[0043] In the time domain, the convolution calculation of the time domain filter and the input image matrix is ​​large and particularly consumes computing resources, so the fast Fourier transform from the time domain to the frequency domain is used to transform the convolution calculation in the time domain into a multiplication operation in the frequency domain, greatly reducing the amount of calculation. The expression formula for the convolution calculation of the time domain filter and the input image matrix is ​​as follows.

[0044] (Formula 1); in, is the response output in the time domain, is the input image matrix, is a time domain filter.

[0045] Formula 1 is transformed into a frequency domain output matrix after fast Fourier transformation.

[0046] (Formula 2); in, is the fast Fourier transform function, is the frequency domain output matrix, is the frequency domain input matrix, is the complex conjugate matrix of the frequency domain filter.

[0047] Obtain the deviation function of the frequency domain input matrix and the frequency domain output matrix under the frequency domain filter. The calculation formula of the deviation function is as follows.

[0048] (Formula 3); in, is the deviation function, is the horizontal index of the frequency domain filter, is the vertical index of the frequency domain filter, For the a frequency domain input matrix with xy dimensions, For the A frequency domain output matrix with xy dimensions, is the complex conjugate matrix of the frequency domain filter with xy dimensions, is the number of frequency domain input matrices, is the total number of rows of the complex conjugate matrix of the frequency domain filter, is the total number of columns of the complex conjugate matrix of the frequency domain filter.

[0049] Desired deviation function The minimum value of is obtained and the deviation function is differentiated. When the first-order partial derivative of the deviation function is 0, the calculation formula of the frequency domain filter is obtained.

[0050] Formula (4); in, is the horizontal index of the frequency domain filter, is the vertical index of the frequency domain filter, For the a frequency domain input matrix with xy dimensions, For the A frequency domain output matrix with xy dimensions, For the The complex conjugate matrix of the frequency domain output matrix with xy dimensions, For the The complex conjugate matrix of the frequency domain input matrix with xy dimensions, is a frequency domain filter with xy dimensions, is the number of frequency domain input matrices, is the total number of rows of the complex conjugate matrix of the frequency domain filter, is the total number of columns of the complex conjugate matrix of the frequency domain filter.

[0051] The initial frequency domain filter at the current moment can be calculated according to Formula 4. The frequency domain filter is a function of time variation, and the initial frequency domain filter at the current moment (moment k) is updated to obtain the first frequency domain filter.

[0052] (Formula 5); in, is the first frequency domain filter with xy dimensions at the current moment (time k), is the filter numerator of the first frequency domain filter, is the filter denominator of the first filter, is the learning rate, is the i-th frequency domain input matrix with xy dimensions at time k, is the complex conjugate matrix of the ith frequency domain input matrix with xy dimensions at time k, is the i-th frequency domain output matrix with xy dimensions at time k, is the complex conjugate matrix of the ith frequency domain output matrix with xy dimensions at time k, is the filter numerator of the second frequency domain filter at the previous moment, is the filter denominator of the second filter.

[0053] Furthermore, the first frequency domain filter at the start time is calculated according to Formula 4.

[0054] For Formula 5, when hour, , for extreme cases Completely by Decision, with the current and It doesn't matter. hour, , for extreme cases Completely by the current and Decide, with It doesn't matter. hour, , for extreme cases Completely by Decision, with the current and It doesn't matter. hour, , for extreme cases Completely by the current and Decide, with It is irrelevant. Obviously, when the surgical robot operating table is in the surgical operation state, the lighting of the surgical environment changes, and the scale and posture of the surgical robot's mechanical arm and end effector change. The second frequency domain filter and learning rate at the previous moment can be used to obtain a more ideal first frequency domain filter at the current moment, which can not only allow the frequency domain filter to quickly adapt to the changes in the target appearance, but also ensure the robustness of the frequency domain filter. When the video image loses frames, there is a large deviation between the first frequency domain filter and the second frequency domain filter.

[0055] This application converts the convolution operation in the time domain into a multiplication operation in the frequency domain through fast Fourier transform, which reduces the amount of calculation. The first frequency domain filter at the current moment is updated according to the second frequency domain filter and learning rate at the previous moment, which not only ensures that the frequency domain filter quickly adapts to the changes in the target appearance, but also ensures the robustness of the frequency domain filter.

[0056] S200: Determine the quality of service (Qos) value of the input image matrix at the current moment based on the first frequency domain filter and the second frequency domain filter, and select the maximum Qos value from a set number of Qos values.

[0057] The quality of service (Qos) value of the input image matrix at the current moment is determined based on the first frequency domain filter and the second frequency domain filter. Specifically, the difference between each first filter value in the first frequency domain filter and the corresponding second filter value in the second frequency domain filter is obtained; the Qos value of the input image matrix is ​​determined based on all the differences and the learning rate.

[0058] The Qos strategy calculation module calculates the difference between each first filter value in the first frequency domain filter and the corresponding second filter value in the second frequency domain filter, and calculates the service quality Qos value of the input image matrix based on all the differences and the learning rate. The calculation formula of the Qos value is as follows.

[0059] ; in, is the first frequency domain filter, is the second frequency domain filter, is the first filter value of the xth row and yth column in the first frequency domain filter, is the second filtered value of the xth row and yth column in the second frequency domain filter, is the learning rate, is the Qos value, is the total number of rows of the frequency domain filter, is the total number of columns of the frequency domain filter, is the difference between the first filter value and the corresponding second filter value in the second frequency domain filter.

[0060] The present application determines the Qos value according to all the differences and learning rates between the first frequency domain filter and the second frequency domain filter, thereby achieving accurate calculation of the Qos value.

[0061] The 5G network QoS control platform sends a set number (N) of different initial QoS policies to the UPF through the signaling message management module, PCF and SMF at different times, so that the UPF can transmit image messages between the surgical robot operation area and the surgical robot control area. At the same time, the 5G network QoS control platform collects image messages at different times through the proximal surgical display system. Different times correspond to different protocol data unit (PDU) dialogues, and calculates the QoS value of the image message at each time, and obtains a total of N (set number) QoS values. Sort the N QoS values, for example, sort the N QoS values ​​in order from large to small to obtain the maximum QoS value.

[0062] S300: Obtaining an optimal QoS strategy for image packets based on the maximum QoS value.

[0063] The optimal QoS strategy represents the best conditions for image message transmission.

[0064] Obtain an initial QoS policy corresponding to the maximum QoS value, for example, obtain the initial QoS policy corresponding to the maximum QoS value from the PDU corresponding to the maximum QoS value, and use the initial QoS policy corresponding to the maximum QoS value as the optimal QoS policy for the image message.

[0065] QoS strategy is the quality standard of network services, ensuring the reliability and performance of image message transmission. QoS strategy includes image transmission indicators such as delay, bandwidth, packet loss rate, and jitter.

[0066] The method for determining the Qos strategy of image messages provided in the embodiment of the present application determines the Qos value through the first frequency domain filter and the second frequency domain filter at two adjacent moments, thereby realizing the monitoring of the transmission quality of image messages at a smaller granularity level in teleoperated surgery, and improving the effect of image quality monitoring in teleoperated surgery. The present application determines the first frequency domain filter according to the input image matrix at the current moment, and conducts in-depth analysis and business monitoring at the image information level, which is conducive to improving the accuracy of determining the optimal Qos strategy. The present application determines the maximum Qos value according to a set number of Qos values, and then obtains the optimal Qos strategy, thereby realizing the provision of the optimal image message forwarding path for teleoperated surgery. The method for determining the Qos strategy of image messages of the present application requires little investment and has good stability, which solves the problem of large investment and serious packet loss in image message quality monitoring in teleoperated surgery.

[0067] Based on the above embodiment, after obtaining the optimal QoS strategy of the image message based on the maximum QoS value, step S400 is further included.

[0068] S400: Send the optimal QoS strategy to the signaling message management module of the 5G network QoS control platform; so that the user plane function module of the 5G network system can control the transmission of image messages from the surgical robot operation area to the surgical robot console and the proximal surgical display system based on the optimal QoS strategy.

[0069] Among them, the signaling message management module is used to send the optimal QoS strategy to the data gateway of the 5G network QoS control platform, the data gateway is used to send the optimal QoS strategy to the policy control function module of the 5G network system, the policy control function module is used to forward the optimal QoS strategy to the session management function module of the 5G network system, the session management function module is used to convert the optimal QoS strategy into a protocol message, and send the protocol message to the user plane function module so that the user plane function module can obtain the optimal QoS strategy.

[0070] like Figure 4 As shown, the optimal QoS strategy selection module of the 5G network QoS control platform sends the optimal QoS strategy to the signaling message management module. The signaling message management module sends the optimal QoS strategy to the policy control function module (PCF) of the 5G network system through the data gateway, and the PCF returns the message of successful delivery of the optimal QoS strategy through the data gateway. PCF forwards the optimal QoS strategy to the session management function module (SMF), and SMF returns a message of successful execution to PCF. SMF converts the optimal QoS strategy into a protocol message, such as the Packet Forwarding Control Protocol (PFCP), and sends the PFCP to UPF through the N4 interface, so that UPF can control the transmission of image messages from the surgical robot operation area to the surgical robot console and the proximal surgical display system according to the optimal QoS strategy. At the same time, UPF returns a message of successful execution to SMF.

[0071] This application sends the optimal QoS strategy to UPF through the data gateway, PCF and SMF, controls the transmission of image messages through UPF, realizes the quality monitoring of the image information business dimension, and is conducive to improving the efficiency of image message transmission in teleoperation surgery.

[0072] Furthermore, the surgical robot console is started based on the following steps: a start instruction is sent to the data gateway based on the signaling message management module, so that the data gateway sends the start instruction to the surgical robot console, and the start instruction is used to start the surgical robot console.

[0073] like Figure 4As shown, the signaling message management module of the 5G network QoS control platform sends a startup instruction to the surgical robot console through the data gateway. The data gateway sends the startup instruction to the surgical robot console. The surgical robot console responds to the startup instruction to start. The surgical robot console returns a successful startup message to the signaling message management module of the 5G network QoS control platform.

[0074] The present application starts the surgical robot console through the start-up instruction of the signaling message management module, thereby realizing the remote start-up and control of the surgical robot console, which is conducive to ensuring the smooth progress of the teleoperated surgery.

[0075] Based on the above embodiments, the 5G network QoS control platform includes a presentation layer, a service and analysis layer, an interface layer and a protocol layer.

[0076] The presentation layer is used to display the optimal QoS strategy, the operation status of the services of the 5G network QoS control platform, and the microservice resources of the 5G network QoS control platform.

[0077] The service and analysis layer is used to orchestrate, modify and delete services of the 5G network QoS control platform.

[0078] The interface layer manages the interface of the 5G network QoS control platform.

[0079] The protocol layer is used to manage the communication protocol of the 5G network QoS control platform.

[0080] like Figure 5 As shown in the figure, the 5G network QoS control platform adopts a layered microservice architecture with a total of 4 layers, including presentation layer, service and analysis layer, interface layer and protocol layer.

[0081] The presentation layer includes a microservice resource presentation unit, an optimal QoS strategy presentation unit, and a business process presentation unit. The presentation unit is used to display various information of the 5G network QoS control platform, including the optimal QoS strategy, the operation status of the services of the 5G network QoS control platform, and the microservice resources of the 5G network QoS control platform.

[0082] The interface layer includes the orchestration interface unit, business interface unit, and microservice interface unit. The interface layer is used to provide various internal and external interface management capabilities.

[0083] The service and analysis layer includes the service orchestration management unit, the service orchestrator execution engine and the microservice management unit. The service and analysis layer is used to orchestrate, modify and delete the services of the 5G network QoS control platform.

[0084] The protocol layer includes the protocol orchestration unit and the communication protocol of the 5G network QoS control platform. The communication protocols include the Robot Operating System (ROS) protocol, the Simple Network Management Protocol (SNMP), the Hyper Text Transfer Protocol (HTTP), etc.

[0085] This application implements the management and presentation of different functions of the 5G network QoS control platform through the presentation layer, service and analysis layer, interface layer and protocol layer, which is conducive to improving the working efficiency of the 5G network QoS control platform.

[0086] The QoS policy determination device for image messages provided in the present application is described below. The QoS policy determination device for image messages described below and the QoS policy determination method for image messages described above can be referenced to each other.

[0087] like Figure 6 As shown, a device for determining a QoS strategy for an image message includes: a frequency domain filter determination module 601, which is used to obtain a first frequency domain filter of an image message at a current moment in a teleoperated surgery and a second frequency domain filter at a previous moment adjacent to the current moment, wherein the first frequency domain filter is determined based on an input image matrix of the image message.

[0088] The maximum QoS value determination module 602 is used to determine the service quality QoS value of the input image matrix at the current moment based on the first frequency domain filter and the second frequency domain filter, and select the maximum QoS value from a set number of QoS values.

[0089] The optimal QoS strategy determination module 603 is used to obtain the optimal QoS strategy of the image message based on the maximum QoS value. The optimal QoS strategy represents the best condition for image message transmission.

[0090] The device for determining the Qos strategy of image messages provided in the embodiment of the present application determines the Qos value through the first frequency domain filter and the second frequency domain filter at two adjacent moments, thereby realizing the monitoring of the transmission quality of image messages at a smaller granularity level in teleoperated surgery, and improving the effect of image quality monitoring in teleoperated surgery. The present application determines the first frequency domain filter according to the input image matrix at the current moment, and conducts in-depth analysis and business monitoring at the image information level, which is conducive to improving the accuracy of determining the optimal Qos strategy. The present application determines the maximum Qos value according to a set number of Qos values, and then obtains the optimal Qos strategy, thereby realizing the provision of the optimal image message forwarding path for teleoperated surgery. The Qos strategy determination method for image messages of the present application requires little investment and has good stability, which solves the problem of large investment and serious packet loss in image message quality monitoring in teleoperated surgery.

[0091] In one embodiment, the frequency domain filter determination module 601 is used to: obtain an output image matrix corresponding to an input image matrix in the time domain based on a time domain filter; perform a fast Fourier transform on the input image matrix to obtain a frequency domain input matrix, and perform a fast Fourier transform on the output image matrix to obtain a frequency domain output matrix; determine a filter numerator of a first frequency domain filter based on the frequency domain input matrix at a current moment, a complex conjugate matrix of the frequency domain output matrix at a current moment, a learning rate at a current moment and a previous moment, and a filter numerator of a second frequency domain filter; determine a filter denominator of a first frequency domain filter based on the frequency domain input matrix at a current moment, a complex conjugate matrix of the frequency domain input matrix at a current moment, a learning rate, and a filter denominator of a second frequency domain filter; and obtain a first frequency domain filter based on a ratio of the filter numerator of the first frequency domain filter to the filter denominator of the first frequency domain filter.

[0092] In one embodiment, the maximum Qos value determination module 602 is used to: obtain the difference between each first filter value in the first frequency domain filter and the corresponding second filter value in the second frequency domain filter; and determine the Qos value of the input image matrix based on all differences and the learning rate.

[0093] In one embodiment, the optimal QoS strategy determination module 603 is also used to: send the optimal QoS strategy to the signaling message management module of the 5G network QoS control platform; so that the user plane function module of the 5G network system controls the transmission of image messages from the surgical robot operation area to the surgical robot console and the proximal surgical display system based on the optimal QoS strategy; wherein the signaling message management module is used to send the optimal QoS strategy to the data gateway of the 5G network QoS control platform, and the data gateway is used to send the optimal QoS strategy to the policy control function module of the 5G network system, and the policy control function module is used to forward the optimal QoS strategy to the session management function module of the 5G network system, and the session management function module is used to convert the optimal QoS strategy into a protocol message, and send the protocol message to the user plane function module, so that the user plane function module can obtain the optimal QoS strategy.

[0094] In one embodiment, the optimal QoS strategy determination module 603 is also used to: send a start instruction to the data gateway based on the signaling message management module, so that the data gateway sends the start instruction to the surgical robot console, and the start instruction is used to start the surgical robot console.

[0095] In one embodiment, the 5G network QoS control platform includes a presentation layer, a service and analysis layer, an interface layer and a protocol layer: the presentation layer is used to display the optimal QoS strategy, the operation status of the services of the 5G network QoS control platform and the microservice resources of the 5G network QoS control platform; the service and analysis layer is used to orchestrate, modify and delete the services of the 5G network QoS control platform; the interface layer manages the interface of the 5G network QoS control platform; the protocol layer is used to manage the communication protocol of the 5G network QoS control platform.

[0096] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the method for determining the Qos strategy of the image message, the method comprising: obtaining the first frequency domain filter of the image message at the current moment in the teleoperation surgery and the second frequency domain filter at the previous moment adjacent to the current moment, the first frequency domain filter being determined based on the input image matrix of the image message; determining the service quality Qos value of the input image matrix at the current moment based on the first frequency domain filter and the second frequency domain filter, selecting the maximum Qos value from the set number of Qos values; obtaining the optimal Qos strategy of the image message based on the maximum Qos value.

[0097] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0098] On the other hand, the present application also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the QoS strategy determination method for the image message provided by the above-mentioned methods, and the method includes: obtaining a first frequency domain filter of the image message at the current moment in the teleoperation surgery and a second frequency domain filter at the previous moment adjacent to the current moment, the first frequency domain filter being determined based on the input image matrix of the image message; determining the service quality Qos value of the input image matrix at the current moment based on the first frequency domain filter and the second frequency domain filter, and selecting the maximum Qos value from a set number of Qos values; and obtaining the optimal Qos strategy for the image message based on the maximum Qos value.

[0099] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the QoS strategy determination method for the image message provided by the above-mentioned methods, the method comprising: obtaining a first frequency domain filter of the image message at the current moment in the teleoperated surgery and a second frequency domain filter at a previous moment adjacent to the current moment, the first frequency domain filter being determined based on the input image matrix of the image message; determining the service quality Qos value of the input image matrix at the current moment based on the first frequency domain filter and the second frequency domain filter, and selecting the maximum Qos value from a set number of Qos values; and obtaining the optimal Qos strategy for the image message based on the maximum Qos value.

[0100] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for determining a QoS strategy for an image message, characterized in that: include: Acquire a first frequency domain filter of an image message at a current moment in a teleoperation surgery and a second frequency domain filter at a previous moment adjacent to the current moment, wherein the first frequency domain filter is determined based on an input image matrix of the image message; Determine the quality of service (Qos) value of the input image matrix at the current moment based on the first frequency domain filter and the second frequency domain filter, and select a maximum Qos value from a set number of the Qos values; An optimal Qos strategy for the image message is acquired based on the maximum Qos value, where the optimal Qos strategy represents the best condition for transmitting the image message.

2. The method for determining the QoS strategy of an image message according to claim 1, characterized in that: The first frequency domain filter is determined based on the following steps: Acquire an output image matrix corresponding to the input image matrix in the time domain based on a time domain filter; Performing a fast Fourier transform on the input image matrix to obtain a frequency domain input matrix, and performing a fast Fourier transform on the output image matrix to obtain a frequency domain output matrix; Determine the filter numerator of the first frequency domain filter based on the frequency domain input matrix at the current moment, the complex conjugate matrix of the frequency domain output matrix at the current moment, the learning rates at the current moment and the previous moment, and the filter numerator of the second frequency domain filter; Determine a filter denominator of a first frequency domain filter based on the frequency domain input matrix at the current moment, a complex conjugate matrix of the frequency domain input matrix at the current moment, the learning rate, and the filter denominator of the second frequency domain filter; The first frequency domain filter is obtained based on a ratio of a filter numerator of the first frequency domain filter and a filter denominator of the first frequency domain filter.

3. The method for determining the QoS strategy of an image message according to claim 1, characterized in that: The determining the quality of service (Qos) value of the input image matrix at the current moment based on the first frequency domain filter and the second frequency domain filter includes: Obtaining a difference between each first filter value in the first frequency domain filter and a corresponding second filter value in the second frequency domain filter; A Qos value of the input image matrix is ​​determined based on all the differences and a learning rate.

4. The method for determining the QoS strategy of an image message according to claim 1, characterized in that: After obtaining the optimal QoS strategy for the image message based on the maximum QoS value, the method further includes: Send the optimal QoS strategy to the signaling message management module of the 5G network QoS control platform; so that the user plane function module of the 5G network system can control the transmission of the image message from the surgical robot operation area to the surgical robot console and the proximal surgical display system based on the optimal QoS strategy; Among them, the signaling message management module is used to send the optimal QoS strategy to the data gateway of the 5G network QoS control platform, the data gateway is used to send the optimal QoS strategy to the policy control function module of the 5G network system, the policy control function module is used to forward the optimal QoS strategy to the session management function module of the 5G network system, the session management function module is used to convert the optimal QoS strategy into a protocol message, and send the protocol message to the user plane function module, so that the user plane function module can obtain the optimal QoS strategy.

5. The method for determining the QoS strategy of an image message according to claim 4, characterized in that: The surgical robot console is started based on the following steps: Based on the signaling message management module, a start instruction is sent to the data gateway, so that the data gateway sends the start instruction to the surgical robot console, and the start instruction is used to start the surgical robot console.

6. The method for determining the QoS strategy of image packets according to claim 4, characterized in that: The 5G network QoS control platform includes a presentation layer, a service and analysis layer, an interface layer, and a protocol layer: The presentation layer is used to display the optimal QoS strategy, the operation status of the services of the 5G network QoS control platform, and the microservice resources of the 5G network QoS control platform; The service and analysis layer is used to arrange, modify and delete the services of the 5G network QoS control platform; The interface layer manages the interface of the 5G network QoS control platform; The protocol layer is used to manage the communication protocol of the 5G network QoS control platform.

7. A device for determining a QoS strategy for an image message, characterized in that: include: A frequency domain filter determination module, used to obtain a first frequency domain filter of an image message at a current moment in a teleoperation surgery and a second frequency domain filter at a previous moment adjacent to the current moment, wherein the first frequency domain filter is determined based on an input image matrix of the image message; A maximum QoS value determination module, configured to determine the quality of service (QoS) value of the input image matrix at the current moment based on the first frequency domain filter and the second frequency domain filter, and select a maximum QoS value from a set number of QoS values; The optimal QoS strategy determination module is used to obtain the optimal QoS strategy of the image message based on the maximum QoS value, and the optimal QoS strategy represents the best condition for transmitting the image message.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for determining the QoS policy of the image message according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for determining the QoS policy of the image message according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for determining the QoS policy of the image message according to any one of claims 1 to 6 is implemented.

Images